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ABSTRACT 

Research sought to develop a quantitative model for 
the prediction of learning effects of a bilingual children's 
educational television program. The stimulus program series consisted 
of 30 sessions of Carrascolendas, a Spanish-English series which was 
based on specified behavioral objectives and aimed at students 
through grade 2. Learning effects were measured with criterion 
referenced tests administered to 408 Mexican-American children. Data 
were collected on the independent variables of communication stimuli, 
individual characte^'istics, and contextual traits; the predictive 
models were based upon linear multiple regression analysis. Results 
showed that learning effects can be predicted with relatively high 
degrees of reliability and accuracy. From this it was concluded that 
policy decisions regarding educational television must account for 
contextual characteristics^ as well as program content and 
objectives, and that a communications stimulus, such as a television 
series, functions as one of many stimuli in producing learning 
effects. More detailed measures of these independent variables are 
needed and contracts for the evaluation of program series should 
require that measurement involve linear models. (LB) 
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Chapter I 

Problem 
Introduction 

The aim of this study was to develop a quantitative model 
for the prediction of learning effects of a bilingual children's 
educational television program series. Both the theoretical and 
practical arguments for this study centered upon the need for a 
predictive model which simultaneously considers the characteris- 
tics of the individual student and the context of communication 
as well as the properties of the instructional communication he 
receives. These arguments were based upon the results of prior 
research on educational television and contemporary theorizing 
on communication, both of which suggest the need for multivari- 
ate models that incorporate mathematical precision. 

The quantitative model in this study was developed through 
linear multiple regression analysis 6f data collected on 408 Mex- 
ican-American children at nine test sites across the nation. This 
data base was assembled during the national evaluation of the bi- 
lingual children's educational television program Carrascolendas . 
Produced by KLRN-TV at the University of Texas at Austin, Carras - 
colendas is aimed at a target audience of kindergarten, first and 
second grade Mexican-American children. It is intended primarily 
for viewing in the classroom as a supplement to bilingual educa- 
tion programs. The series ^^onsists of thirty half -hour programs 
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broadcast in color durinq tht^ course of a sinqle semester. Each 
provjram is dcsivjnod to promote loarniruj of pro-specified instruc- 
tional objectives. The programs consist of a number of generally 
active r fast-paced segments of songs ^ dramatic skits, animated 
sequences r and other techniques structured to hold the viewer 
child's interest and stimulate learning of the concepts being 
presented. The series is unified by a central cast of characters 
and the setting of an imaginary town from which the program series 
derives its name, Carrascolendas . 

Research Questions 

Given the instructional communication materials and the 
data base identified above r this study attempted to answer the 
following questions: 

(1) To what degree can le^arning effects be predicted 
upon the basis of characteristics of the instructional communica- 
tion treatment r the individual student characteristics ^ and the 
contextual variables? 

(2) If a reliable model for predicting learning effects 
can be developed: (a) What are the practical implications for 
policy decisions on children's bilingual educational television? 
(b) What are the implications for theorizing about the communica- 
tion effects of educational television? 

A Paiadigm for Communication Research 



ERLC 



The current trend toward multivariate research on 



3 



coirmiunication is reflective of a general pattern in social and 
behavioral science researcli. Within the framework of cost-ef- 
fectiveness and governmental funding during the past decade of 
social programs such as Head Start and Model Cities^ a great 
deal of research has concentrated on program evaluation (Alex- 
ander, 1972; Rossi & Williams, 1972). Re^^earch on these pro- 
grams has evolved from simple notions about cause-effect to mul- 
tivariate views. Poverty r for example, was once cited as the 
direct cause of crime; "^programs which raised income levels should 
consequently lower crime rates. Such has not been the case 
(Alexander, 1972, p. 140). Alexander points out that theorists 
and researchers alike have begun to realize that a fundamental 
inadequacy in early research was a failure to discern, measure, 
and account for the "multitudinous interacting influences on so- 
cial behavior [p. 133) 

Evaluation and accountability have also had a heavy im- 
pact in education (Coleman, et al., 1966). The contracts for 
educational television programs supported with government funds, 
such as Sesame Street and Carrascolendas , more often than not cur- 
rently require empirical evaluation of the impact of the program* 
This evaluation research suffers from the same problems of account- 
ing for multivariate influences as research on the evaluation of 
other types of social programs. Unfortunately the traditional 
research paradigms, developed in experimental laboratories and 
usually incorporating analysis-of-variance designs, have not been 
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entirely adequate for the task. 

The research paradi<.7m adopted in the present study incor- 
porated at the outset the assumption that learning effects pro- 
duced in part by educational television are also influenced by a 
variety of receiver and contextual characteristics. In addition ^ 
the statistical technique of multiple regression analysis was as- 
sumed to provide an adequate mathematical tool for the construc- 
tion of a multivariate predictive model. 

Communication Theory and Research 

Contemporary theorizing on mediated communication views 
the media as operating amid a nexus of causes in producing ef- 
fects (De Fleurr 1970; Rogers & Shoemaker, 1971; Schramm & Rob- 
erts r 1971) . This view has evolved from an extensive body of 
research conducted principally on attitudinal effects of medi- 
ated communication. Excellent , extensive reviews of this re- 
search are available , particularly in Weiss (1969) and Roberts 
(1971). The key stages in the development of the multivariate 
view of comiT^unication and attitude change are summarized here be- 
cause these theories have often been applied to research on learn- 
ing effects of educational television. The multivariate view of 
communication represents considerable change from earlier simple 
stimulus-response approaches • 

The earliest models used by behavioral scientists in 
their research on communication was basically a stimulus-response 
theory r embodied in the "hypodermic needle** model of communication. 
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The media were viewed as a simple stimulus which directly acted 
upon a passive, homogenous audience of receivers to produce the 
effect ♦ This direct effect view of communication, however, was 
too simple and too superficial to account for the variety of dif 
ferent responses found by communication researchers (Rogers & 
Shoemaker, 1971) . At first theorists attempted to account for 
differences in response to communication by recognizing Individ^ 
ual psychological differences* This slight change in theory was 
still inadequate. Eventually cx variety of social variables was 
incorporated into views of the communication process, and this 
was known as the "two-step" flow model of communication (Rogers 
^ Shoemaker, "1971/ p. 205). in this model the mass media were 
seen as influencing opinion leaders, who in turn influenced less 
active members of the audience. The two-step flow model quickly 
evolved into the multi-step ^low model and generally into a theo 
ry of mediated communication which accounted for a variety of so 
cial categories and social relationships. The consistent gener- 
alization threading throughout current theorizing on mediated 
communication is that receiver and contextual variables cypicall 
interact with message and channel variables in producing effects 

Research and theorizing on educational television has 
generally lagged somewhat behind research on attitude change . 
Until the late 1960 's, research on educational television con- 
sisted primarily of comparative studies of televised as against 
conventional instruction. These studies mostly followed the 
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hypodermic needle model of conununica tion . One group of students 
was exposed to an educational television program/ and another 
group to a classroom teacher. The measure of effects was gen- 
erally the subject area test customarily administered to the 
class of students under consideration . Responses to the effects 
measure were tested to determine whether the stimuli produced 
significantly different effects. The most frequent finding of 
425 comparative studies reviewed by Schramm (1962) was that 
classes taught by educational tele^'ision revealed no significant 
difference on subject area tesLs from classos taught by conven- 
tional methods. VJhere significant differences were found, how- 
ever, there was seldom any clear indication of the reasons for 
these results. After many comparison studies had been accumu- 
lated and reviewed, Schramm (1962) concluded that television was 
generally found to be more successful among the lower elementary 
grades than in high school or college studies, and television was 
more frequently successful in studies of science or mathematics 
instruction than in studies of instruction in the humanities or 
literature. In reviewing many of the same studies, Williams 
(1962) suggested that the attitude, ability, and personality of 
the teacher might be an important missing variable in accounting 
for the learning effects of educational television. 

Two main conclusions have generally emerged from the com- 
parative studies (Chu & Schramm, 1968) : (1) Educational televi- 
sion could be as effective a medium for teaching as conventional 



methods. (2) Older students (high school and college level) 
generally express unfavorable attitudes toward educational tele- 
vision. These broad summary- findings give some indication that 
the impact of educational television is influenced both by audi- 
ence and contextual variables as well as by the content of the 
television program. Theorists such as Allen (1971) , Gagne (1970) , 
Kittross (1969) , and Schramm (1962) , have stressed repeatedly the 
need for multivariate research on educational television. 
Schramm's (1962) statement, although now more than a decade old^ 
is still relevant to contemporary research on educational televi- 
sion: 

The pertinent question is no 2onger whether the teacher 
can teach effectively on television, but rather how, 
when, for what subjects, and with what articulation into 
classroom activities [educational] television can mo::e 
effectively be used [p. L65] . 

In answer to the need for theory and research on educa- 
tional television which accounts for the multivariate nature of 
communication and learning, the current state of the art is quite 
limited. Only generalizations linking two or three variables are 
possible, such as the statement that the use of visual images 
will improve the learning of manual tasks (Chu and Schramm, 1968). 
As De Fleur (1966) has observed, generalizations stated in terms 
such as "will improve," "may alter," and "often" need to be re- 
placed by statements of "appropriate mathematical functions and 
statements of probability [p. 139]." Kittross (1969) and Schramm 
(1971) have also emphasized the importance of models which provide 
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for tlie prediction of outcomes bar^oJ upon observable relationships 
and conditions. While the construction of such models has not 
been attempted in research on educational television, a statisti- 
cal tool useful for the construction of the needed models exist 
in multiple regression analysis. 

Multiple Regression Analysis 

The essential concepts of multiple regression analysis 
provide insight into the applicability of this technique in the 
present study. One of the most basic concepts is that a subject's 
scores on one measure can be reliably predicted from his scores on 
other measures. 

The reasoning in the present study is that performance 
test scores can be predicted from a weighted combination of mea- 
sures of communication stimuli, individual receiver characteris- 
tics, and contextual characteristics. For example, a student's 
test score might be expected to vary according to the number of 
educational television programs he views, his grade level, and 
the amount of bilingual instruction he is given. In natural 
language it might be said that these factors influence the 
student's response to test items and thus contribute to or de- 
tract from his test score. If information about these factors is 
known, then the student's tost score could be predicted from this 
information. 

In mathematical terms, the prediction equation would be 
of the form: 



9 



Test Score = (a,) (Viewini Frequency) 
^ (a2) (Grade Level) 

+ (a^) (Bi linquil Instruction (hours/day] ) 
+ (Error) 

In this equation the a^, a^r and a^ are the weights on the three 
measures used to predict test score. These measures are select- 
ed examples of communication stimuli r receiver characteristics r 
and contextual characteristics. The error term is the difference 
between the predicted test score and the test score actually ob- 
served. Multiple regression analysis is used to reduce the data 
from a sample of subjects to a single generalizable equation. 

The basic research paradigm of the present study was one 
of model-seeking. The goal of the analysis was the construction 
of a mathematical equation representing the optimum multivariate 
model for predicting specified communication effects. The ef- 
fects of interest in the present study were the stated objectives 
or effects expected to result from viewing the educational tele- 
vision program series Carrascolendas . Selection of independent 
variables used to predict effects was guided by the results of 
prior research on Carrascolendas and a similar series , Sesame 
Street. 

Prior Research 

Sesame Street Research 

In 1968 the Children's Television Workshop began an inno-- 
vative program of research ^ production ^ and evaluation for the 
educational television program Sesame Street (Palmer^ 1969). One 
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key assumption was that the producers of Sesaaie Street accepted 
at the outset the burde*. of gaining the attention and interest of 
their target audience. Thi.^ represented a departure from the vast 
majority of prior educational television projects, which general- 
ly assumed a captive classroom audience. The research activities 
connected with Sesame Street were an integral part of the devel- 
opment, production, and evaluation of the progr<*m series. De- 
parting from the assumptions and methods of comparative studies 
of educational television, the aims of Sesame Street were iden- 
tified in terms of educational objectives and such responses as 
attention, motivation, comprehension, and the like. These ob- 
jectives were stated as specified behaviors which could be read- 
ily observed and measured. Evaluation research and production 
on Sesame Street also attempted to take account of both audience 
and content variation (Ball & Bogatz, 1970). The target audience 
was defined generally as preschool children, aged 3-5, and more 
specifically as disadvantaged, urban children. The behavioral 
goals of the program were originally formulated with reference 
to the target audience, but some formative research was conducted 
to determine baseline data. These data were then used as refer- 
ence points for deciding whether the initial statement of behav- 
ioral goals was realistic or was inappropriately high or low. 
Given refined statements of the behavioral objectives, the tc^k 
of evaluation was then one o£ selecting samples of children and 
developing and applying instruments which gauged the specified 
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behaviors. 

First year research. The first summative evaluation of 
Sesame Street included the following main question (Ball & Bogatz, 
1970, p. 6) : 

1. Do viewers learn more than nonviewers? 

2. Among viewers, which children learn least, and which 
children learn most? 

3. What is the progrcun's effectiveness 2unong subgroups 
(e.g., lower-middle class, boy-girl, Spanish-English 
speakers) ? 

The evaluation design involved sampling four populations 
of children, with emphasis on the first type: 
1. Disadvantaged; inner city 
2 • Advantaged ; suburban 
3. Advantaged and disadvantaged; rural 
4 • Disadvantaged; Spanish-^&^^eaking 

Test site selections were limited by a number of criteria, 
including: (1) the area had to be served by a VHP (rather than 
UHF? educational television station; (2) the station had to agree 
to telecast Sesame Street at 9:00 or 10:00 a.m.; (3) the site had 
to contain a large number of disadvantaged, inner city children; 
(4) there had to be a local, competent field staff availab^^. 
The five sites selected were Boston, Massachusetts; Durham, North 
Carolina; Phoenix, Arizona; suburban Philadelphia, Pennsylvania; 
and a rural area in northeastern California. 
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In these sites suh jcH'ts woro selected L'rom lload Start 
classes (in-school subjects) and at-home children, primarily on 
the basis of the child's socio--economic status. Within each sam- 
ple, children were randomly assigned to treatment conditions, en- 
couraged or not encouraged to view Sesame Street and observed or 
not observed by field staff while viewing. Although 1124 chil- 
dren participated in the evaluation, only 943 completed both pre- 
test and posttest. 

The tests had undergone an extensive period of develop- 
ment and were criterion referenced, i.e., the tests were designed 
to measure the behaviors which were the specified co\x^ of the 
program series. Attempts were made to provide a familiar envir- 
onment for testing with a relatively short test r'uration. The 
general technique consisted of three steps. A picture or object 

4 

v/as shown to the child; the picture was described; and the child 
was asked to respond (usually by pointing) to a question about 
the picture or object. 

Results of the first year's evaluation may be summarized 
as follows: 

1. Viewers learned more during the progress of the pro- 
gram series than nonviewers. 

2. Consistent viewers learned far more than occasional 
viewers, with at-home viewers suffering no disadvantage in com- 
paris>rn with in-school viewers. 

3. Amount of viewing did not interact with sex, that is 
girls did not gain more or less than boys at each viewing level. 

ERLC 
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4, Differences in gains for children from different so- 
cial status were tied to prior achievement. Prior achievement 
was, in turn, confounded with amount of viewing. Children with 
high pretest levels generally gained less, since they already 
knew much of the program's material. Children with low prior 
achievement benefited most from increased viewing of the program. 

A separate analysis was conducted on data from 4 3 Mexi- 
can-American children who viewed Sesame Street at home in Phoe- 
nix, Arizona. Since some of these children had relatively lower 
fluency in English, the children were tested, when necessary, in 
Spanish by bilingual test interviewers. Results were not ana- 
lyzed, however, in terms of the language of the test interview. 
These children were further divided into qy^fttiles according to 
their frequency of viewing Sesame Street. Children were assigned 
to one of the four quartiles as follows: 

(1) Children who rarely or never watched the program 
(N=14) . 

(2) Children who watched an average of 1 to 3 times a 
week (N^ll) . 

(3) Children who watched an average .of four to five 
times a week (N=7) . 

(4) Children who watched an average of more than five 
times a week (N-ll) . 

For all quartiles, pretest scores were low as compared 
with other groups of children in the national evaluation. The 
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gains of Mexican-American children in quartile 4 (high frequency 
of viewing) gained an average of 37% from pretest to posttest. 
This gain was larger than that of any other group in the national 
evaluation. As with other groups, increased frequency of view- 
ing resulted in greater gain from pre to posttest. The more fre- 
quently the child viewed the program, the greater was the pro- 
gram's impact. 

Frequent viewers did not differ appreciably from infre- 
quent viewers in terms of mental age or socio-economic status. 
Frequent viewers, howpver , more often had parents who usually 
read to them and had mothers who talked with them about Sesame 
Street . These related or supplementary activities appeared to 
be more important moderating influences on the impact of viewing 
the program than mental age or socio-economic status. 

These conclusions were considered tentative, since the 
number of Mexican-American children (4 3) who participated in the 
study was relatively small. An effort was made in the second 
year to replicate the results of the first year's testing (Bo- 
gatz & Ball, 1971). A group of 66 children from Spanish back- 
grounds was sampled in Los Angeles and randomly assigned to en- 
couraged and not-encouraged treatment conditions. For unreport- 
ed reasons, however, the treatment and control distinctions broke 
down, with nearly all of the children viewing the program series. 
Results were inconclusive and no replication of the findings of 
the first year was possible. 



The results of the first year's evaluation not only pro- 
vided a basis for defining the success of the program, but were 
further useful in diagnosing the first year's effort. For exam- 
ple, the evaluation raised such concerns as whether the curricu- 
lum was a poor preparation for learning to read; whether the 
goals of the program were too narrowly conceived, thus more great 
ly assuring success, whether the program was adequately oriented 
to the culture of Spanish-speaking children, and whether sequence 
which attempted rote learning could be improved. 

Second year research . Evaluation during the second year 
consisted of a "New Study" and a "Follow-up Study" with the fol- 
lowing questions (Bogatz & Ball, 1971, p. 29) : 

1. Can the findings of the first year be replicated? 
(The "New Study") 

2. What are the effects of the revised sites and curri- 
culum? (The "New Study") 

3. Were the first year's effects cumulative? (The "Fol- 
low-up Study" — longitudinal) 

The New Study concentrated mainly on at-home, urban, dis- 
advantaged children, representing the main target population. 
Tvo new sites included 283 children who had not seen Sesame 
Street during its first year. The Follow-up Study retained three 
disadvantaged urban sites (283 children). New items and tests 
were developed to assess the new goals; old items and tests were 
included to measure the effectiveness of old goals on new viewers 
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Aqain, tho study iiivolvod tostinq group:; of rliildrcn who could 
be divided into viowor-nonvicvvor populations. 

The results of the second year evaluation for the New 
Study supported the first year findings of goal effectiveness for 
disadvantaged preschool children. Goal effectiveness is described 
as impact on, rather than perfect mastery of, goal behaviors. The 
general results revealed that disadvantaged preschoolers contin- 
ued to benefit from the show's emphasis on basic knowledge and 
skills, and older and more advantaged children benefited from the 
presentation of more complex knowledge and skills. Results of 
the Follow-up Study showed greatest effects in the cognitive do- 
main. Since most of the first year's material was learned in the 
first year, it was deemed necessary to introduce new goal areas 
each year in order to have a continuing educational intact on 
children viewing the series for more than one year. 

A dividend of the research on the Sesame Street series 
has been that it provides a basis for expanding the kinds of 
questions which may be raised about the effects of educational 
television, as well as making such questions more precise. For 
exaunple, one question raised by the results of the Sesame Street 
evaluation concerns the effects of the viewing environment upon 
learning gains. Children who viewed Sesame Street and had the 
opportunity to interact with their mothers about the program dem* 
onstrated greater gains than other children. The child seemed to 
become a more active educational participant if circumstances 



17 

prompted him to use items learned from the program, such as the 
use of related toys or some type of interchange with parents or 
peers. A more general question is how, with any type of educa- 
tional series, supplementary activities can be encouraged at home 
or at school and what influence such activities have on learning 
effects resulting from viewing the program series. 

An expansion of questions raised in the evaluation of 
Sesame Street concerns the effects of viewing an educational 
television program series on Mexican- American children. Research 
on the first year of Sesame Street indicated that among Mexican- 
American children, increased frequency of viewing the series re- 
sulted in increased learning effects, and that these effects were 
somewhat moderated by environmental influences. However, the un- 
fortunate failure of the replication attempted during second year 
research leaves the answers somewhat inconclusive. 

Carrascolendas Research 

Again, the educational television program series Carras - 
colendas was designed specifically to meet the needs of bilingual 
kindergarten, first, and second grade Mexican-American children. 
It was initiated in July, 1970, as a special bilingual program 
project of the Education Service Center^ Region XIII (Austin, 
Texas) . The content of the series was designed at the outset to 
reinforce and supplement existing classroom bilingual instruction- 
al programs. Produced by the University of Texas at Austin 



18 



educational television 5tatioi\, KL,RN-TV, Carrascolcndas has in- 
corporated a variety of presentational techniques such as films, 
puppets, and dramatic skits and is similar in format to S esame 
Street. Also like Sesame Street , the producers of Carrascolendas 
accepted the burden of attracting and maintaining t ; attention 
of their target audience. The content of the progrdm series was 
developed around specified behavioral objectiv^^c. For two years 
Carrascolendas was broadcast (and evaluated) only in the state of 
Texas. In its third year the series was broadcast nationally 
through the Public Broadcasting System. 

In the first two years of evaluation research on Carras - 
colendas , only Mexican-American children in an Austin, Texas, 
school were included in experimental studies (Natalicio & Williams, 
1971; Williams, McRae, & Van Wart, 1972). In the third year re- 
search, when Carrascolendas was nationally distributed, subjects 
were drawn from test sites in Arizona, California, Colorado, and 
Michigan, in addition to Texas (Williams, Van Wart, & Stanford, 
1973) . 

The same type of research design was used in the experi- 
ments for each of the three years. Test instruments were devel- 
oped to measure a sampling of the behavioral objectives of the 
series. Groups of children were selected to participate in the 
experiment and were designated as viewers or nonviewers. Before 
viewing the series all children were given oral interview tests, 
separately in Spanish and in English. Viewers then watched the 
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series which urually consisted of 30 half -hour programs broadcast 
three times a week. After viewing the series, both groups of 
children were given the same test materials. Gain scores (post- 
test minus pretest) were compared to see whether there were dif- 
ferences between viewer and nonviewer groups. Differences, when 
found, were interpreted as indicative of the effects of the pro- 
gram. 

In all three years, behavioral objectives were specified 
for several content areas. ^ Analyses were conducted on subtesc 
scores in each content area, as well as on total test scores for 
both Spanish and English testing. The content areas were tested 
in both languages as follows: 

Self Concept 

Science (Labeled as Physical Environment during the first 
year) 

History /Culture (Labeled as Multicultural Social Envi- 
ronment during the first year) 

Language Skills 

Math (Labeled as Concept Development during the first 
year) 

In addition, one other content area was tested only in Spanish — 
Phoneme/Grapheme Relations. 

The first two years — Austin , Texas, evaluations . In the 



Detailed examples of each content area are given in 
Chapter Two. 
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first year's experiment Mexican-American children from bilingual 
and traditional classrooms in first and second grades from one 
Austin, Texas, school were randomly selected for assignment to 
viewer or nonviewer groups. Eighty-eight children were included 
in this study, 4 8 viewers and 40 nonviewers* Subdivisions by 
grades and type of classroom were equal within the viewer and the 
nonviewer groups. Children in the nonviewer group were provided 
with an alternative activity in the form of a photography pro- 
ject« The viewing group received a selected set of activities 
before the program, viewed the program, and then usually engaged 
in follow-up activities. In both pre and posttesting, approxi- 
mately half of the children received the English interview test 
before the Spanish one, and vice versa, to avoid order effects. 
The pre and posttest interviews conducted with each child were 
always separated by a period of several days. 

Research conducted during the second year's evaluation 
was basically the same as during the first year. Children in the 
same Austin, Texas, school were again selected for participation 
in the evaluation experiment. In the second year experiment, how- 
ever, the 44 children were selected only from the first grade and 
were randomly assigned to one of three treatment groups: (1) 
viewers who had supplementary classroom activities relevant to 
the program series; (2) viewers who had no classroom activities 
directly relevant to the program; and, (3) a group of children 
who were nonviewers. Again, the nonviewer group was engaged in a 
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photography class during the period that the Carrascolendas pro- 
grams were on the air. Test instruments used during the second 
year were also criterion referenced measures. These test instru- 
ments represented samplings of the behavioral objectives of the 
second year program series, which was a new series different from 
that broadcast during the first year. 

Table 1 summarizes the results of the first two years of 
evaluation research. 

The results of the first year's experiment revealed sig- 
nificant viewer-nonviewer differences in English testing, but not 
in Spanish testing (Natalicio & Williams, 1971, pp. 29-33). The 
findings were generalizable across traditional and bilingual 
classrooms and across first and second grades. Results were dif- 
ferent, however, across content areas, with significant differ- 
ences found only in English testing of Science and Math. 

The general results of the second year's experiment were 
reported in terms of scores representing the percentage of cor- 
rect responses made by subjects (Williams, McRae, & Van Wart, 
1972, pp. 14-15, 19-21) . As can be seen in Table 1, a varying 
pattern of results is again exhibited across content areas. 

Scoring procedures used during the second year were some- 
what expanded from those used during the first year. Whereas in 
first year testing the children's responses were scored only as 
right or wrong, in the second year responses were also weighted 
to indicate degree of correctness, as follows: (1) a correct. 
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Table 1 

Results of the First Two Years 
of Carrascolendas Evaluation 



First Year Second Year 



Gpanish Areas 

Self Concept * 
Science 

History/Culture 

Language Skills (Spanish) * 
Math 

Phoneme/Grapheme 
Total 

English Areas 

Self Concept 

Science * 

History/Culture * 
Language Skills (English) * 
Math * 

Total * * 

♦Significant viewer-nonviewer difference (p<. 05). 



ERIC 



23 



umprompted response was weighted "4"; (2) a correct, buv prompted 
response was weighted "3"; (3) a partly correct response received 
a "2"; and, (4) an incorrect or no response was given a "1". 

Additional analyses were conducted on Math items and hc- 
tivity-nonactivity items, usiig average scores which ranged from 
one to four. An average scor<5 was computed for each child based 
upon the weighted scores for relevant items. The varying results 
of testing in different languciges is particularly obvious from the 
analysis of average Math scores (Table 2) , where in Spanish test- 
ing viewers without activities exceeded ocher groups in gains, 
while in English testing viewers with activities were superior. 
A similar pattern is evident in the analysis of activity-nonac- 
tivity items (Table 3) , although the gains of viewers with activ- 
ities generally exceed other groups. 

The results of the analysis of scores on Math items and 
on activity-nonactivity items taken together indicate that the 
language of the test interview is an important consideration in 
research on educational television for Mexican-American children 
^nd that supplementary activities as well as program viewing pro- 
duce learning effects among the children. 

The third year — national evaluation . The^generality of 
learning effects produced by program viewing was shown in research 
conducted on a national basis during the third year of Carrasco - 
lendas evaluation activities- Again, the research design, test- 
ing, scoring, and analysis procedures were generally the same as 

ERIC 



24 



Table 2 

Average Scores in Response to 
Math Items in Second Year Testing 



Posttest 
Scores 



Difference from 
Pretest Scores^ 



Spanish Test: 

Viewers with activities 
Viewers without activities 
Nonviewers 



2.6 
2.6 
2.4 



0.5 
0.9 
0.5 



English Test: 

Viewers with activities 
Viewers without activities 
Nonviewers 



3.1 
2.8 
2.8 



0.8 
0.4 
0.5 



^Significant difference in gains among groups (p^.05) 
in both Spanish and English testing. 



Table 3 

Average Scores in Response to Content Items 
Involving or not Involving Class Activities 



Posttest Difference from 
Scores Pretest Scores 



Activity Items ^ 



Spanish Test: 

Viewers with activities 2,7 0,5 

viewers without activities 2,6 0.6 

Nonviewers 2,3 0.1 

English Test: 

viewers with activities 2,9 0.7 

viewers without activities 2.6 0,3 

Nonviewers 2.6 0,3 



Nonactivity Items 



Spanish Test: 

Viewers with activities 2.8 0.5 

Viewers without activities 2.7 0.4 

Nonviewers 2.7 0.2 

English Test: 

Viewers with activities 2.9 0.7 

Viewers without activities 2.6 0.3 

Nonviewers 2.4 0.1 



^Significant difference in gains among groups (p<.05) 
in both Spanish and English testing of Activity Items. 
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used in the second year evaluation. Subjects for the third year 
experiment, however, included Mexican-American children in the 
following test cities: 

Albuquerque, New Mexico 

Edinburg , Texas 

Lansing , Michigan 

Los Angeles, California 

Pueblo, Colorado 

San Antonio, Texas 

Tracy , California 

Tucson, Arizona 

In this national evaluation, Mexican-American children were se- 
lected from kindergarten as well as first and second grades and 
were assigned to viewer or nonviewer groups. The Pueblo site 
had an additional group of Anglo subjects in kindergarten, first, 
second, and third grades who received the posttest measurement 
only in Spanish, although both pre and posttests in English were 
administered. If any subject did not view at least twenty of the 
first 27 programs (only objectives presented in the first 27 pro- 
grams were tested), he was eliminated from the analysis. Elimi- 
nation of these subjects reduced the total number of viewers post- 
tested for statistical comparison. The experiment was completed 
by a votal of 340 viewer and 106 nonviewer subjects in Spanish 
and 341 viewers and 106 nonviewers in English. 

Two sets of analyses were conducted on data collected dur- 
ing the third year's experiment. One set of analyses was conducted 
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on total test scores and content area subtest scores combining 
viewers or nonviewers frcm all sites (Table 4) • The other anal- 
yses were conducted on total test scores of viewer-nonviewer 
groups at each test site (Table 5) . 

These results indicate that children in different grades 
respond differently to the program series, and that children in 
different g3ographic locations across the nation respond differ- 
ently to items presented on the program series (and subsequently 
tested) in Spanish or in English. The influence of the viewing 
environment on learning effects 2unong children, noted above in 
Sesame Strt^et research, is evident in the third year Carrascolen - 
das research by the differential response of children in differ- 
ent cities. The environmental factors which produced this dif- 
ferential response were not investigated in the third year Carras - 
colendas evaluation; but, the results of Sesame Street research 
and the second year Carrascolendas research point strongly to the 
variable of supplementary activities as one influential environ- 
mental factor. 

The results of analyses of responses according to content 
area also confirm the findings of earlier evaluations. There is 
a differential pattern of responses among content areas, with the 
most salient gains in the content area of History/Culture and the 
least in Phoneme/Grapheme Relations. 

Viewed across the three years of evaluations, the results 
exhibit a pattern sensitive to content area, grade level, school 
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Table 4 
Third Year National Results 



Test 


Kinder- First Second Combined 
garten Grade Grade Grades 


Spanish 


Self Concept 
Science 

History/Culture 
Language Skills 
Math 

Phoneme /Grapheme 
Total 


★ 

* * * * 

* * 

* * 

* * 


. English 


Self Concept 
Science 

History/Culture 
Language Skills 
Total 


* 

* * 

* * * 


*Viewer gains 

(p<.05) • 


significantly greater than nonviewer gains 
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differences, and language of testing. Where effects arc measured 
relative to the behavioral objectives of the program series, the 
series appears to have its greatest impact in the area of History/ 
Culture and Language Skills among first grade viewers who parti- 
cipate in supplementary classroom activities relevant to the pro- 
gram content. The total test score appears to be an ambiguous 
measure, whereas content area subtests yield more specific in- 
formation. Significant viewer-nonviewer differences, furthermore, 
are found more frequently in English than in Spanish testing. 

Given that effects of viewing an educational television 
program can be measured and that such effects vary according to 
program content, viewer characteristics, and school differences, 
the question is to what extent are effects predictable from such 
information? If effects can be reliably predicted, then what are 
the more salient predictive factors? The collection and analysis 
of data aimed at answering these questions is discussed below.. 



Chapter II 

Program Materials and Data 

Descript : on of Program 

The program materials represented in this study consisted 
of the third-year series of Carrascolendas , This series was orig- 
inated as a special bilingual education proj'^^^t supported by Title 
VII funds. The series was produced by KLRN-TV ' ^ the University 
of Texas at Austin under the production supervision of Ms. Aida 
Barrera. 

The content of Carrascolendas was designed to reinforce 
and supplement existing classroom bilingual instruction programs, 
while taking advantage of the unique opportunities offered by the 
television medium. The series incorporated a ^road range of pre- 
sentational techniques such as actors r puppets ^ films , music, 
Jance, and special effects to provide innovative , attractive, and 
entertaining enrichment of classroom activities. In the develop- 
ment of program material, emphasis was placed on bilingualism, 
Mexican-American culture, the Mexican and Spanish heritage and 
the multi-cultural society of the United States. Throughout the 
series the Spanish language, Mexican-American actors, Mexican- 
American personalities, music, games, dances, rhymes, and other 
elements of Hispanic tradition reflected the bilingual-bicultural 
environment of the target audience. 

Consonant with the aims of th^ existing bilingual 
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education programs and classrooms , the primary qoal of Car ras- 
colendas was to /acilitate the children's knowledge gains in se- 
lected first and second grade content areas, and to increase 
their usage skills in both the English and Spanish languages. 
Ms. Barrera and her production staff designed materials for each 
of the 30 half -hour programs to promote learning of specified be- 
havioral objectives in each of the selected content areas. Ob- 
jectives were provided by a Curriculum Coordinator at the Edu- 
cation Service Center (Region XIII) and approved by an Advisory 
Committee. The behavioral objectives for the series were devel- 
oped within six knowledge or content areas. The following cap- 
sule descriptions illustrate these areas with different examples 
in Spanish and English: 

1. Self Concept — knovjing about one's self, recognizing 
cultural heritage, and knowing suitable ways to express emotions. 

(Examples : ) 

iQue idiomas habias? 

(What language do you speaK?] 

Name someone, a person or an animal, who needs you to 
help him. 

2. Science — skills such as identifying life cycles of 
certain animals, knowing processes such as making bread, and be- 
ing able to identify environmental sights and sounds. 

cCulil se rompe facilmente? 

(piedra, madera, hierro, vidrio) 
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[Which breaks easily?] 

(rock, wood, iron, glass) 

Put these pictures in order to show how honey is made. 

3. History-Culture — knowing legends, songs, and facts 

of United States and Mexican history. 

cComo se le dice a las cascaras de huevo que tienen 
confetti? 

[What do you call egg shells filled with confetti?] 

What are pinatas used for? 
(visual - a pinata) 

4. Language Skills — knowing certain words in Spanish and 
English, selected syntactic patterns, and sound contrasts in Span- 
ish and English. 

cComo se le dice a la persona que hace pan? 

(What do you call the person who makes bread?] 

This is a picture of a mouse. In this picture there arc 

two of them. There are two . 

(visual - 2 mice) 

5. Phoneme - Grapheme Relations — knowing relationships be- 
tween sounds and their corresponding letters, as a skill relevant 
to reading. 

iCual de estas letras es la /k/? 
(qu, X, z, r, b) 

[Which of these letters is the /k/?] 

6. Math — learning certain sets, counting, and symbol 
recognition. 



cQue hora es? 

(reloj - 3:00) 

[What time is it?] 

(picture of a clock showing 3:00) 

Color one-third of this ball. 

(sheet - ball divided into thirds) 

Based upon behavioral objectives such as these, each pro- 
gram was comprised of a variety of segments, some running only a 
few seconds, others up to two or three minutes. Each segment fo- 
cused on a single behavioral objective. No attempt was made to 
introduce a particular segment or to have special transition ma- 
terial between segments. However, when segments were combined 
for a thirty minute program, considerable attention was given to 
the "flow" from one segment to another, to variety in types of 
segments, and to the inclusion of logically ordered behavioral 
objectives. These kinds of segments — with a single objective or 
two, highly interesting, and often including a "character" — 
seemed to benefit from the personalized and intense focus offered 
by the television medium. Evidence of the potential of this ap- 
proach with young children has also been provided by the success 
of Sesame Street . Continuity for each program and for the series 
was provided by setting most of the segments in the imaginary 
town of Carrascolendas, and by the regular appearance of four or 
fi/e main characters throughout the series. 

In creating the content for Carrascolendas , Ms. Barrera 
has drawn heavily from her own bilingual and bicultural experiences. 
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Her strategy has been to create segments in which Spanish is used 
as it is in everyday life. In a special television segment de- 
veloped to introduce the program serios to teachers ^ Ms, Barrera 
commented: 

...we present Spanish as a language that is natural and 
acceptable. Because television is essentially a dramatic 
medium, we rely heavily on dramatic techniques 'co get our 
point across.... We tiy to play up the comical aspects and 
we try to do it in a picaresque fashion. We feel the pic- 
aresque is especially appropriate, becav.se the roots of 
this dramatic form are Spanish-speaking ones. 

The picaro was exemplified by one of the most popular 
regular characters in Carrascolendas , "Agapito" the lion. Other 
regular characters who appeared in dramatic skits, dance segments, 
and songs included "Senorita Barrera;" the "Professor;" "Herman," 
an actor who portrayed a marionette; Spanish and English-speaking 
hand puppets; and "Caca," a young Mexican-American girl. In ad- 
dition to the regular cast, numerous children from the Austin, 
Texas, area appeared on the programs and prominent Mexican-Amer- 
icans were featured in cameo roles. 

Presentational techniques ranged from elaborate animation 
and location filming of scenes such as a glassware factory to a 
single character holding up a sign with a vowel on it. The em- 
phasis in all techniques was on fast pacing and high interest ap- 
peal for children. The third year's series was produced in color 
to take advantage of the appeal of bright, changing colors for 
young children. 

A typical program included the following: A brief segment 
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in Spanish in which Agapito delightedly pops four balloons, one 
by one, to illustrate the concept of zero; — a dramatic skit pre- 
senting the prepositions "in" and "on" in which the Professor, 
Herman, Srta. Barrera, ar d the children ride "in" and "on" large 
cardboard cutouts of a trai.i, bus, and car;--an animation sequence 
showing a stork trying to shelter himself from the rain with a 
very tiny umbrella while the words cigiiena (stork) and paragiiitas 
(umbrella) are repeated; --an English song, "The World Is Round," 
introduced by a film of a rocket launch; and — a quick dance freeze 
i'^ v^hich a joke is told in Spanish. 

Although Carrascolendas used almost equal amounts of Span- 
ish and English, the Spanish segments were usually shorter than 
the English ones and oftei. consisted of rhymes, riddles, games, 
and jokes* Every program began with a song or an animated se- 
quence, which have been found in previous evaluation to be the 
two most popular aspects of the program. The format for Program 
19 in the third year's series provides examples of the duration 
of each segment, the language used, and the order of the segments 
(Figure 1) . 

Like any television script, the script for Carrascolendas 
includes descriptions of characterization, sets, music, props, 
and the like. But beyond this, each script contains descriptions 
of the behavioral objectives for each segment, so that production 
staff and performers are aware of the educational aims underlying 
what they are preparing. An example is the script for the 
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Time 


Seg 


ment 




0:47 


1 . 


Intro 




0 : 30 


2 . 


Filler 


: Dance Freeze #8 


0 : 21 


3 . 


Math: 


Disappearing Balloons 


n m A 1 

U : 4 1 


A 

4 . 


Animation GU /gw/ 


1:44 


5. 


Song : 


The World Is Round (REPEAT) 


2:59 


6. 


Film: 


The Truck 


0:20 


7. 


Filler 


: Adult Cameo #1 


0:19 


8. 


Cameos 


: GU /gw/ 


0:30 


9. 


Filler 


: Dance Freeze # 5 


1:27 


10. 


Song : 


Stand Everybody Side by Side 



Language 

Spanish 
Spanish 
Spanish 
English 
English 
Spanish 
Spanish 
Spanish 



(REPEAT) English 

3:58 11. Dramatic: The Falling Ball Spanish 

1:01 12. Animation G /g/ (REPEAT) Spanish 

0 ; 30 13. Filler: Dance Freeze #11 Spanish 

2 :17 14. Puppets: Willy and Gravity English 

1:31 15. Song: Naricita, Naricita Spanish 

0 : 33 16. Math: Subtracting Lights Spanish 

0:27 17. Filler: Adult Cameo #4 — La Navaja 

(The Knife) Spanish 

1:01 18. Animation GU /g/ (REPEAT) Spanish 

1 ;18 19. Dramatic: In/On English 

1:44 20. Song: La Vieja Ines (REPEAT) Spanish 

0 ;14 21. Math: Disappearing Balloons Spanish 

0 : 30 22. Filler: Dance Freeze #13 Spanish 

0:41 23. Animation: GU /gw/ (REPEAT) Spanish 

2:07 24. Song: B/V Song (REPEAT) English 

0 : 30 25. Filler: Dance Freeze #9 Spanish 

0:20 26. Close 

28:20 

Figure 1. Sample Program Outline 
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dramatic segment "In/On" of Prc^gram 19 shown in Figure 2. 

To facilitate classroom use of Carrascolendas , a package 
of supplementary materials was dt^veloped and distributed as a 
Teacher Guide. The guide contained activities and songs coo^"- 
dinated with television segments of Carrascolendas , as v;ell as 
additional activities not tied to the program. The guide was 
deemed necessary to encourage teachers to provide children with 
"doing" as well as "viewing" experiences. Because the television 
presentation could not always include all facets of each behav- 
ioral objective, the guide offered an expansion of concepts and 
ideas. This made it possible for the teacher to encourage sup- 
plementary classroom activities, which would have been difficult 
to illustrate in the series. 

For the monolingual English-speaking teachers, the Teach- 
er Guide provided insight into the Mexican-American culture. It 
helped the teacher see the rationale behind the behavioral objec- 
tives. In this way it was hoped that Carrascolendas activities 
would become closely tied and pertinent to the regular classroom 
curriculum. 

For instance, the Teacher Guide description of the "In/On ' 
segment of Program 19 includes an explanation of the difficulty 
Spanish speakers have with these English prepositions (Figure 3) . 
Suggestions for songs / games / and other activities were also in- 
cluded in the guide. While fulfilling the essential behavioral 
objective criterion, each activity included in the guide was 
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(Two clovns pMsh in cutout of rar. Ftop &j.d hold two side^ of it. They face front. 
The ProfeoBor and children scranolt: to get on it. The Frofoscor sh>*s: Come on... We're 
going for a ride. We're going for a ride. There's a pause after they get settled. The 
Profesaor speaks) " 

The P: We're in the car. We're in the car. 
Children: We're in th^ car* We're in the car. 
Key: In (flashes) 

(Sound effect of car leaving, apatterinr Jerkily .., '^lovn.': will nove the car back and 
forth to the tirae of the Jerks. ..car -^ontin^ies to ^f^utter. . . ii«»3 . The Frofesaor atarta 
to wave. . .stops, corA.'*; out to front ot -jar . . .raisev, V .oi — ho^ctVlly, can aee saoKe 
eoBing out or sor<» such— •ria:/be clowr. cm *»»t becic there an.* plant snoke bos*... the Pro- 



fessor turns to the audience) 
The Pt Well, I gueus ve didn't gc for a ride after aJl. 



fifur* 2. ^aaple 5^ript of Carra'^col endaa S^^'.nx, 
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HNGLISH LANGUAGK SKILLS - Ta and On 

This segment shows children on a bus, on a train, 
on the floor, on a chair, and in a car. 

Uses of "on" in English are difficult for Spanish 
speakers who generalize, "If 1 can say 'on a bus,' 
'on a bicycle," 'on a train,' I can say 'on a 
car.'" This last expression would have a very 
different meaning from "in the car" which is what 
a native speaker of English would say. 



Figure 3. Teacher Guic ^ Description of In/On Segment of 
Program 19 
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chosen with a view to intrinsic interest^ stimulation, humor, 
and fun for the children. 

An evaluation of usage of the Teacher Guide indicated 
that teachers were generally pleased with the guide and were in- 
corporating its use into their classroom curriculum. 

Selection of Test Sites and Subjects 

The subjects for this study were selected from among Mex- 
ican-American children in grades kindergarten througli second who 
participated in the 1972-73 national evaluation of the bilingual 
educational television program Carrascolendas . In the national 
evaluation eleven test sites in eight cities were chosen as rep- 
resentative of different types of Mexican-American populations 
across the nation. 

Test Sites 

Test cities and sites were chosen by the Carrascolendas 
evaluation team in cooperation with the Education Service Center, 
Region XIII, and were as follows: 
Albuquerque, New Mexico 
Edinburg , Texas 
Lans i ng , Michigan 
Los Angeles, California 
Pueblo, Colorado 
San Antonio, Texas 
Tracy, California 
^ Tuc son, Ar i zona . 
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Although data from subjects in Pueblo, Colorado ^ were included in 
■ha national evaluation, these data were not included in the pre- 
sent study because the information necessary to match classroom 
and student data was not provided by the Pueblo field consultant. 

Two test sites were chosen in each of the most populous 
states in terms of Mexican-American communities, California and 
Texas, to obtain results from both urban and rural areas. The 
Michigan site represented children of migrant families. Further, 
Albuquerque and San Antonio each had two test sites to obtain re- 
sults from children involved in a bilingual classroom curriculum 
and children involved in a monolingual (English) curriculum. Most 
sites had bilingual programs which were funded by Title VII. 
Those schools not having bilingual programs were located in Lan- 
sing and Tracy. 

The average school enrollment at test sites was 556, with 
the smallest school in Lansing (215), and the largest in San An- 
tonio (972) . All sites with the exception of Lansing and Tracy 
had an enrollment of 75% or more Mexican-American children. At 
Lansing, the average enrollment of Mexican-American children was 
30% and at Tracy it was 24%. Of the 13 test site schools involved 
in the evaluation, the approximate annual income of families in 
10 was $1,000-4,999. The remaining three schools, located in 
Lansing and Tracy, had approximate annual family incomes of $5,000- 
9,999. Time of viewing Carrascolendas varied among the sites, 
with most children viewing the series between 9:00 a.m. and 12:30 
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p.m. In Albuquerque, with an air time of 2:30 p.m., teachers in- 
dicated displeasure with afternoon viewing of the series. Class- 
es in San Antonio and Tucson were the only ones who viewed Carras- 
colendas in color; all others viewed th<i series in black and 
white. 

Subjects 

Mexican-American children in kindergarten, first and sec- 
ond grades at each of the eleven test sites were selected for 
participation in the Carrascolendas national evaluation. As 
stated above, however, children at the two test sites in Pueblo, 
Colorado, were not included in the present study. Field consul- 
tants attempted to select subjects randomly from among children 
in classes viewing or not viewing Carrascolendas . The number of 
classes not viewing the series, however, was limited and nonview- 
er subjects in each grade at each test site tended to come from 
a single class. At each test site 15 viewers and 5 nonviewers 
in each grade were originally pretested. Posttests were to be 
completed with 12 viewer and 4 nonviewer subjects fror every 
grade level to allow for illness and/or transfers from the school. 
The final numbers of subjects who completed the test sequence is 
given for each site and grade level in Table 6. A total of 405 
subjects completed testing in Spanish, with 406 subjects complet- 
ing testing in English. As can be seen in Table 6 , nearly equal 
numbers of viewers completed testing in each grade level at each 
site. Similarly, nearly equal numbers of nonviewers completed 
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Table 6 

Numbers of Subjects by Site and Grade Level 



Site 




Viewers 






.'mvicwers 








K 


1st 


2nd 


K 


1st 


2nd 


Albuquerque , 




12^ 


12 


12 


4 


4 


3 


Albuquerque r 


NM^ 


12 


12 


12 


4 


4 


4 


Edinburgr TX 




12 


12 


12 


4 


4 


4 


Lansing, MI 




1 n 

X u 


e 

9 


lid 


3 


0 


4 


Los Angeles r 


CA 


13^ 


15 


13 


5 


4 


5 


San Antonio r 


TX* 


12 


12 


12 


5^ 


5= 


3 


San Anton iOr 


TX^ 


12 


12 


12 


0 


0 


0 


Tracy r CA 




11 


11 


12 


4 


4 


4 


Tucson r AZ 




11 


12 


11 


4 


4 


4 


Grade Subtotals: 


105 


103 


107 


33 


29 


31 


Viewer /Nonviewer 
Subtotals 

0 




315 






93 




Total Subjects 






408 








Total subjects who 


completed 


testing 


in Spanish 


• 


405 




Total subjects who completed 


testing 


in English 


• 


406 





^Site with bilingual curriculum 
*^Site with monolingual (English) curriculum 
^One subject in this group did not complete testing in 
Spanish. 

^One subject in this group did not complete testing in 
Q English^ 
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testing in each grade level at each site, with the exception of 
one of the sites in San Antonio. At this site nonviewers in mono- 
lingual (English) curriculum classrooms could not be obtained. 

Data Collection 

Measures of the variables of interest in the present 
study were collected with a variety of instruments as follows: 

(1) Pre and postseries criterion referenced tests pro- 
vided measures of learning effects resulting from 
viewing the Carrascolendas series; 

(2) Reports prepared by teachers of children who parti- 
cipated in the study provided measures of: 

(a) Individual viewer characteristics? 

(b) Contextual variables; and, 

(c) Characteristics of commnnication stimuli. 
Data collection in each test city was coordinated by a 

field consultant. The consultants were recruited and trained 
during the summer (1972) prior to broadcast of the series. Con- 
sultants met with the evaluation team during the second week in 
August, 1972, to review the evaluation activities. They viewed 
a sample program of Carrascolendas and were supplied with sample 
instruments as well as detailed procedures for hiring testers 
and for instructing testers in interview and scoring techniques. 
During the evaluation experiment, members ol the evaluation teaun 
were available to all field consultants upon request. 

O 
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In addition to coordinat in^j testinq of the children par- 
ticipating in the experiment and providing information regarding 
the test site, field consultants obtained absence reports for 
children participating in the experiment and distributed other 
instruments for completion by the teachers. The various instru- 
mentr used to collect data for the present study are described 
in further detail below. 

Criterion Referenced Tests 

The learning effects of the Carrascolendas program series 
were assessed in terras of preseries and postseries measurements. 
These measurements were taken on criterion referenced tests, where 
test items were developed from Scunplings of the instructional ob- 
jectives of Carrascolendas . The curriculum staff for the Educa- 
tion Service Center, Region XIII, prepared the original set of 
objectives which was presented to the production staff for incor- 
poration in the series. The evaluation team sampled from among 
those objectives presented in the series and developed test ques- 
tions on the saanpled objectives. Two tests were developed, one 
in Spanish the other in English, and each test was admi?;istered 
in an oral interview by a tester fluent in the test language. 
The details of test instrument construction and administration 
are discussed below. 

Instrument development . The criterion referenced instru- 
ments were based upon behavioral objectives specified prior to 
program production. A total of 357 objectives were specified in 



several content areas by the Education Service Center staff. Of 
these, the production staff incorporated 269 objectives in the 
series and the test instruments tested 103 of these. Table 7 
illustrates the relative breakdown of objectives into content 
areas both by nun^er and by time allotted in the program scries 
to objectives presented. Of the content areas. Phoneme /Gr apheme , 
Spanish Language Skills, and Math were treated exclusively in 
Spanish, with English Language Skills treated exclusively in 
English. History/Culture, Self Concept, and Science were treat- 
ed in both Spanish and English. While most objectives were treat- 
ed in one language only, some objectives were treated twice, once 
in Spanish and once in English. Both the English and the Spanish 
versions of the test instruments, however, were designed to cor- 
respond with objectives treated only in Spanish or only in Eng- 
lish. A total of 71 objectives was tested in Spanish and 32 ob- 
jectives were tested in English. 

The development of individual test items is best illus- 
trated by a specific exeunple. For instance , one of the objec- 
tives of the series was that after being exposed to a story about 
a celebration, the child would be able to identify related pic- 
tures taken directly from the program, such as a picture of a 
pinata. This objective was tes^e^ in the English test with two 
questions: (1) What is this? (The child is shown a picture of 
a pinata.) (2) What are pinatas used for? 

The Spanish test instrument consisted of 56 items, while 
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the English test consisted of 44 items, Tho same questions were 
used on pretests and posttest<^, but the questions were reordered 
foi posttest administration. These instruments were revisions of 
instruments used in the previous regional evaluation of Carrasco- 
lendas conducted in 1971-72. As a result of revision, 3/ items 
on the English test were retained from the previous year, with 
36 items being retained on the Spanish test instrument. The 
tests used in the national evaluation (197 2-73) were analyzed 
to determine the reliability of the entire test, as well as the 
reliabi? ity of the subtests for each content area on both the 
Spanish and the English test instruments. One useful measure of 
the consistency of a test is the alpha coefficient of reliabil- 
ity. Tnis coefficient, which can range from 0 to 1.0, gives an 
estimate of such qualities of the test as whether the test dif- 
ferentiates high scorers clearly from low scorers and whether the 
test is internally consistent. Alpha coefficients of reliability 
were computed for the total test and for the content area sub- 
tests and are displayed in Table 8. As can be seen from Table 8, 
the reliability of the entire test was high on both pre and post 
administration of both the English and Spanish tests. Reliability 
coefficients of the content area subtests, as expected due to re- 
duced test lengths, ranged from a low of .51 to a high of .91. 

Admini.">tration and scoring procedures . Pretesting at 
most test sites was completed prior to initial viewing of the 
series. In Lansing, however, pretesting was not completed until 
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Table 8 

Alpha Coefficients of Reliability 
of Criterion Referenced Test Instrnj:ients 



Self History I^guage Phoneme 
Concept Culture Skills Science Grapheme Math Total 

Spanish 



Number of 



items 


5 


8 


14 


4 


13 


12 


56 


Pre 


.81 


.91 


.68 


.53 


.87 


.90 


.96 


Post 


.80 


.89 


.81 


.51 


.85 


.87 


.96 



English 



Number of 



items 


6 


6 


16 


16 


44 


Pre 


.63 


.80 


.51 


.84 


.90 


Post 


.66 


.83 


.55 


.86 


.92 
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after the first three programs in the series had been shown. 
Some children at the Lansing site, therefore, may have viewed 
one or two programs before they received pretests • All posttest:- 
ing was completed prior to the December holidays in order to elim* 
inate the potential contamination of effects due to differential 
retention of material learned. Posttesting was begun at some 
sites prior to airing of the last program in the series. Al- 
though some children may not have seen the last few programs in 
the series prior to receiving the posttest, only objectives pre- 
sented in the first 27 programs were tested. Thus, very few if 
any children were tested on objectives from a program they had 
not seen. 

Childrer at all sites were tested individually in Span- 
ish and English with three or more days elapsing between admin- 
istration of the tests in each language. The tests were admin-- 
istered in oral interviews by individuals who were fluent in the 
test language and had been trained by field consultants in the 
interview techniques to be used. It was recommended that all 
testers spend time in the classrooms prior to testing and that 
they conduct two or three sample interviews with children who 
were not subjects in the evaluation experiment. According to 
reports from testers these procedures were followed. 

While conducting the test interviews, the tester met with 
the child outside his regular classroom. In most schools tests 
were administered ^n relatively private areas. Efforts were made 
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throughout the interview to maintain a relaxed atmosphere, with 
all remarks addressed to the child in the language in which the 
test was being conducted. 

Each question was read aloud to the child and any neces- 
sary visual materials were presented by the testers. The child's 
response was scored by the tester on the test instrument. A for- 
mat of the test instrument is illustrated by the first page of 
the English and the Spanish tests presented in Appendix A. 

Responses to test items were scored in two ways: content 
and language usage. Content scores were differentiated as follows 
a perfect response (weighted as a "4"), a prompted response 
(weighted as a "3"), a partial response (weighted as a "2")^ and 
an incorrect or no response (weighted as a "1") . Language usage 
scores were not considered in the analysis in the present study. 

The test instruments were designed with numeric scores 
for content and language on each item (Appendix A) • The inter- 
viewer circled the number which defined the child's response. 
These numbers were then coded to provide data for computation of 
individual scores • 

In order to determine the accuracy of scoring among test 
interviewers an instrument was developed to measure the reliabil- 
ity of scoring. An Interviewer Reliability Form consisting of 
six test questions was sent to each tester (Appendix B) . Each 
question also gave a hypothetical child's response. The inter- 
viewer was required to mark the score he/she would give the child 
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on each test question. This instrument did not require the iden- 
tification of the interviewer in hopes that the form would be 
completed without distortion. Forms were identified by site, how- 
ever^ by checking postmarks (whenever possible) upon receipt. 

The results provided evidence of generally reliable uni- 
formity among sites. Two Spanish field agents in Albuquerque 
scored at least half cf the items wrong, however, making results 
from that site somewhat questionable. The Edinburg, Lansing, and 
San Antonio sites each had two responses marked wrong. A few re- 
sponses were received which could not be identified according to 
site due to missing postmarks. 

Evaluators reviewed all instruments prior to coding, and 
those items requiring the child to respond by marking on the test 
instrument were checked against scored results; adjustments were 
made if necessary. 

Dependent variables . Each child had a variety of scores 
whicn represented his responses in the Spanish and English inter- 
views. The child's average response on an entire test represents 
an overall measure of learning effects which resulted from view- 
ing the whole program series. Accordingly r a total average score 
was computed for each child from his responses to all items on a 
particular test. This score is a complex measure however, since 
several content areas were treated in the program. The process 
of computing a total average score vitiates the differences in 
the impacts of the several content areas included in the total 
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average. In order to see the effects in each of the content 
areas, subtest scores were also computed using the child's re- 
sponses to items pertaining to a particular content area. The 
following scores were computed for each child's responses to the 
Spanish and English pre and posttests: 

(1) total average score; the average score on a one-to- 
four scale on all items on a particular test. 

(2) Self Concept score; the average score on items per- 
taining to the content area of Self Concept. 

(3) History/Culture score; the average score on items 
pertaining to the content area of History/Culture. 

(4) Language Skill items; the average score on items 
pertaining to the content area of Language Skills. 

(5) Science; the average score on items pertaining to 
the content area of Science. 

Two additional scores were also computed for the Spanish 
interview test on content areas which were treated only in Span- 
ish: 

(6) Phoneme/Grapheme score; the average content score of 
items pertaining to the content area of Phoneme/ 
Grapheme Relations. 

(7) Math score; the average content score of items per- 
taining to the content area of Math. 

Site and Teacher Reports 

As noted above, the independent variables of interest in 



this study were characteristics of the communication stimuli, in- 
dividual viewer characteristics, and contextual characteristics. 
The only information collected on individual viewers was the grade 
level and the sex of each subject. This information was obtained 
from the test interviewers. Teachers of children who participated 
in the Carrascolendas national evaluation provided information re^ 
garding their own backgrounds, classroom activities, student ab- 
sences, and communication stimuli. Information about communica- 
tion stimuli included the number of Carrascolendas programs viev.'ed 
by each subject and whether the programs were viewed in color cr 
black and white. In addition, information was obtained as to 
whether the subject also viewed Sescune Street and/or any other 
educational television programs. Information regarding the view- 
ing of other educational television programs was sought because 
it provides a perspective on Carrascolendas viewing. If the sub- 
ject, for example, viewed no other educational television pro- 
grams, then for that subject viewing of Carrascolendas was a some- 
what isolated instance of instructional stimulation through edu- 
cational television. On the other hand, if the subject viewed 
other educational television programs, then viewing of Carrasco - 
lendas could be considered less a novel event and more a practice 
of some familiarity to the student. Viewership of Sesame Street 
was isolated as an inrlependent variable for two reasons — first, the 
1972-73 Sesame Street programs were targeted in part at an audi- 
ence of Mexican-American children; and, second, although Sesame 
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Street was targeted at a younger audience than the Carrascolendas 
audience, a number of teachers reported that subjects participat- 
ing in the Carrascolendas experiment also viewed Sesame Street as 
a regular part of their classroom activity. Information about 
contextual variables included measures of the teacher's background 
and fluency in Spanish, the presence and Spanish fluency of a 
teacher aide, the amount of bilingual instruction conducted in 
the classroom, whether supplementary activities related to Carras - 
colendas were conducted in Spanish or English, and the nuniber of 
children and ethnic composition of the class. Instruments used 
to collect information on these independent variables are dis- 
cussed in further detail below. 

Site reports . Field consultants were asked to provide a 
Site Status Report between the times pretesting was completed and 
posttesting was begun. A copy of the form used for these reports 
is contained in Appendix A. The information from item 7 regard- 
ing the broadcast dates of individual Carrascolendas programs was 
used in comparison with absence reports (Appendix A) provided by 
classroom teachers in order to determine the actual number of pro- 
grams viewed by each subject. 

Absence reports ♦ After the broadcast of the Carrascolen - 
das series had been completed for the fall semester, each teacher 
of students who participated in the national evaluation was given 
a form listing the names of viewer or nonviewer students in her 
class (Appendix A) . Field consultants distributed these forms to 
the teachers and requested that the teachers list all of the 
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dates that each student was absent during the semester. As 
stated above, these dates were then compared with program broad- 
cast dates in order to determine the number of programs actually 
viewed by each student. 

Classroom information reports , Classroom information was 
obtained on a form completed by each teacher and illustrated by 
the examples in Appendix A. Two sets of classroom information 
forms were used: one for teachers of nonviewer students and one 
for teachers of viewer students. On the nonviewer classroom in- 
formation form, the teacher reported her fluency in Spanish, her 
academic degrees, and the number of years she had taught school. 
The teacher also reported the amount of bilingual instruction 
given to her class and whether or not the class participated in 
any televised instruction. The teachers response to the ques- 
tion, "Do you speak Spanish?" was coded as follows: English only 
was coded as a 0; limitedly as a 1; moderately as a 2; and, flu- 
ently as a 3 . Whether the teacher had a bachelor's degree and 
master's degree were both coded as binary variables , weighted as 
a 1 if the teacher had the degree and 0 if not. Years of teach- 
ing experience was coded directly as a continuous measure. The 
amount of bilingual instruction was recorded as three measures, 
weekly frequency of bilingual instruction, for example, was 
weighted as a 5 for daily instruction to a 1 for instruction once 
a week. The number of hours per day of bilingual instruction was 
coded as a continuous measure. The number of hours per week of 
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bilingual instruction was also computed by multiplying frequency 
times hours per day* This computation provides a kind of lump 
sum measure of bilingual instruction. Participation in televised 
instruction was an open-erded question on the form. In coding 
responses, mentions of Sesame Street were coded as a 1 in a bi- 
nary variable, and the number of educational television programs 
mentioned r other than Carrascolendas ^ was recorded as fx continu- 
ous variable. The number of children in the class r also report- 
ed on this form, was recorded as a continuous measure. 

In addition to the information requested of teachers in 
nonviewer classrooms, teachers of viewer students reported whether 
they initiated supplementary activities related to Carrascolendas 
before or after the program, and whether the activities were con- 
ducted in Spanish or English. Supplementary activities and the 
language of activities were recorded as binary measures, i«e«r 
if supplementary activities were conducted a 1 was scored, 0 
otherwise. Two measures were used to code the language of activ- 
ities, one for Spanish and one for English. The coding was accom- 



plished as follows: 

Spanish English 
Mainly Spanish 1 0 
Mainly English 0 1 
Half Spanish/Half English 1 1 
No activities 0 0 



Teacher Diary. Most of the teachers of viewer students 
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maintained a program-by-proqram *'diary" of conimonts and ratinq 
scales evaluating each program in the series. The first page of 
the Teacher Diary was a form requesting general information (Ap- 
pendix A) • Data regarding the presence in the classroom of a 
teacher aide and whether the aide spoke Spanish were recorded in 
binary form, 1 if yes and 0 if no. The ethnic composition of 
the class was also reported on the general information form. The 
percentage of Mexican-American students in the class was record- 
ed for use in the present study as a continuous variable ranging 
from 1 to 100. 

Independent variables . Data collection during the course 
of the national contract evaluation of Carrascolendas yielded the 
following measures of independent variables available for analy- 
sis in the present study: 

(1) Pretest scores which were measures of the knowledge 
levels of the subject prior to viewing any of the program series. 

(2) Measures of communication stimuli, including frequen- 
cy of viewing of the Carrascolendas program series; whether the 
program was viewed in color or not; viewing of Sesame Street; and, 
viewing of other educational television programs. 

(3) Measures of individual characteristics, including 
grade level and sex. 

(4) Measures of contextual characteristics, including 
measures of supplementary activities (in Spanish and in English) ; 
measures of the amount of bilingual instruction given the 



subjects in terms of frequency, hours per day, and hours per 
week; measures of the background (s) of the teaching staff, in- 
cluding whether the teacher had a bachelor's degree and a mas- 
ter's degree, the teacher's experience and Spanish fluency, 
whether a teacher aide was present in the subject's class, and 
whether the aide spoke Spanish; and, the number of children and 
percentage of Mexican-^American children in the subject's class. 



Chapter III 



Analysis and Results 
Overview 

The primary goal in this study was the development of a 
mathematical model for the prediction of learning effects of a 
bilingual children's educational television program series. As 
discussed in Chapter Two, the program material consisted of the 
Carrascolendas educational television series. Learning effects 
were defined as scores on criterion referenced tests administered 
after broadcast of the entire program series. Independent vari- 
ables included measures of viewership of the program series, 
other communication stimuli, individual receiver characteristics, 
and contextual characteristics. 

The statistical technique of linear multiple regression 
analysis was utilized for the construction of a predictive math- 
ematical model. The strategy for constructing an optimum model 
was as follows: 

(1) Dependent and independent variables were selected 
and measures were defined -in terms of multiple regression nota- 
tion. 

(2) The subject pool was divided into two equal samples. 
One of these samples was used in prevalidation construction of 
the model. The remaining sample was used in cross-validation of 

61 



the model. 

(3) Independent variables which consisted of continuous 
measures were tested to determine whether they had a significant 
curvilinear relationship with the dependent variable in a parti- 
cular model* 

(4) A starting model was constructed for each dependent 
variable. This model included all potential independent vari- 
ables, along with curvilinear expressions where appropriate. 

(5) The starting model was constructed using an itera- 
tive procedure. This procedure yielded estimates of the error 
mean square at each step in the construction of the model. The 
estimates of the error mean square were visually inspected to de- 
termine which independent variables contributed least to the pre- 
diction equation. These variables were initially eliminated as 
the next step toward the construction of an optimum model. 

(6) A series of restricted models was constructed by 
successively eliminating remaining independent predictor variables 
which contributed least to the prediction equation at each step. 

(7) These restricted models were applied to the cross- 
validation sample to determine which model provided optimum pre- 
dictive efficiency. 

The details of the analysis and results are presented in 
order below. 



Selection and Definition of Variables 



Dependent Variables 

The dependent variables available for analysis in this 
study consisted of total average scores on criterion referenced 
tests administered in Spanish and in English, and subtest scores 
on content areas. Since the present study aimed at predicting 
learning effects resulting from viewing the Carrascolendas pro- 
gram series r dependent variables which were reliably and consis- 
tently related to program viewing were chosen for analysis. The 
selection of such dependent variables thus permitted investiga- 
tion of the correlates of program viewing in the attainment of 
learning effects. 

Table 9 presents the correlations of the viewing measure 
with posttest scores for the total tests and the content area 
subtests in both Spanish and English. As can be seen in the 
table r the Spanish subtest of History/Culture and the English 
subtest of Science were the only test scores significantly corre- 
lated with the measure of viewership of the program series ^ with 
Spanish History/Culture scores exhibiting the strongest correla- 
tion. The English subtest of Language Skills and the English 
Total test scores also approached significant correlation with 
the viewing measure (a correlation of .1380 was required) . As 
pointed out in Chapter Twu^ the content area subtests represent 
less ambiguous measures than the total test^ since the total test 
score was computed by averaging across items from all content 
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Table 9 

Correlations of Viewing Scores 
and Posttest Scores 



Name of Posttest 


Correlation 


with Viewing Measure 


Spanish 


Self Concept 




-.0349 


Language Skills 




.0135 


History/Culture 




.1939* 


Science 




-.0038 


Phoneme/Grapheme 




.0893 


Math 




.0475 


Total 




.0597 


English 


Self Concept 




.0076 


Language Skills 




.1220 


History/Culture 




.1132 


Science 




.1487* 


Total 




.1291 



*p<.05 
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areas* The English Total test score, therefore, was eliminated 
from consideration as a dependent variable in the present study. 

The results of the national contract evaluation analyses 
were presented and discussed in Chapter One (Tables 4 & 5) • These 
analyses indicated that significant viewer-non/iewer differences 
were found most consistently in Spanish testing of the History/ 
Culture concept area and next in English testing of Language 
Skills. Significant results were indicated for kindergarten and 
combined grades in English testing of Science. The reliability 
of the English subtest of Language Skills, however, was among the 
lowest for any content area, whereas the reliabilities for the 
other two subtests were quite high, with the Spanish History/ 
Culture subtest the highest of any content area (see Table 8, 
Chapter Two) . 

Since the Spanish History/Culture subtest was the subtest 
which was most reliably and consistently associated with program 
viewing, this measure was chosen as the primary dependent variable 
of interest for the construction of a predictive model. Based 
upon its high reliability and significant correlation with the 
viewing measure, the English Science subtest was also chosen as 
a dependent variable for the construction of a model to provide 
comparison and contrast with the model constructed for the pre- 
diction of Spanish Hi story /Culture scores. 

In the notation of multiple regression analysis, the de- 
pendent variable will hereafter be referred to as the criterion 
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variable. Since one criterion variable is dealt with at a time 
in multiple regression analysis, separate models were constructed 
using Spanish History/Culture posttest scores and English Science 
posttest scores as criterion variables. 

Independent Variables 

The independent variables available for analysis in this 
study included measures of communication stimuli, individual 
ch**racteristics , r.nd contextual characteristics. The collection 
o£ data on these variables is discussed in detail in Chapter Two 
and will not be repeated here. Nineteen independent variables 
were available for use in the present study. ^ Because a sub- 
ject's posttest score depends to a certain extent upon his pre- 
test score, the pretest scores in Spanish History/Culture or Eng- 
lish Science were added to the independent variable set for the 
respective models predicting posttest scores. 

In the notation of multiple regression analysis, measures 
of the independent variables will be referred to as predictors , 
and the pool of subjects* scores on a particular variable is re- 
ferred to as a vector. For example, the list of all subjects" 
scores on the Spanish History /Culture posttest is a criterion 
vector for one of the models constructed in this study. The 



The measure of whether the subject's teacier had a bach- 
elor's degree provided no differentiation aunong subjects nor pre- 
dictive power, since all of the teachers had a bachelor's degree. 



corresponding scores of those subjects on the measure of viewer- 
ship of the Carrascolendas program series is one of the predictor 
vectors in a predictor set. 

The criterion and predictor vector sets are summarily 
listed and defined in Table 10. 

Division of the Subject Pool 

The subject pool (408 subjects) was divided into two 
equal samples, so that models constructed with one sample could 
be croris-validated with data from the other sample. The div^ision 
was accomplished by selecting every other subject from each tost 
site for inclusion in one sample, with the remaining subjects in- 
cluded in the other S£unple. In the following discussion, the sam- 
ple used for original construction of models is referred to as 
the prevalidation sample. The other sample is referred to as the 
cross-validation sample. 

Spanish History /Culture Model Construction 

Although the procedures used for the model construction 
were the same for both criterion vectors, construction of models 
for each will be discussed separately in order to avoid confusion 
of the results of statistical tests conducted at different steps 
in model construction. 



Linearity Tests 

The relationship of a predictor variable with the criterion 
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Table 10 




Variable Set 




Name 


Definition 


Criterion 


Posttest 


A vector containing the posttest scores of 
the subjects. 


Predictors 


Unit 


A "unit vector" all of whose elements are 1 


Pretest 


A continuous vector containing the pretest 
scores of the subjects. 


Viewing frequency 


A continuous vector containing the number 
of Carrascolendas programs viewed by the 
subject (ranges from 0 to 27) . 


Color TV 


A binary vector containing a 1 if the pro- 
gram was viewed in color, 0 otherwise. 


Sesame Street 


A binary vector containing a 1 if the sub- 
ject viewed Sesame Street, 0 otherwise. 


Educational TV 


A continuous vector containing the number 
of educational television programs, other 
than Carrascolendas, viewed by the subject. 


Grade 


A continuous vector containing the subject' 
gcade level. 


Sex 


A binary vector containing a 1 if the sub- 
ject i3 feinale, 0 otherwise. 


Activities (any) 


A binary vector containing a 1 if the sub- 
ject was given supplementary activities, 0 
otherwise 
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Table 10 (continued) 



Name 



Definition 



Predictors (continued ) 



Activities (Span. ) 

Activities (Eng.) 

Bilingual instruction 
(days) 

Bilingual instruction 
(hours/day) 

Bilingual instruction 
(hours/week) 

Teacher MA 

Teacher experience 

Teacher Spanish 
Aide 



A binary vector containing a 1 if supple- 
mentary activities were conducted in Span- 
ish, 0 otherwise. 

A binary vector containing a 1 if supple- 
mentary activities were conducted in Eng- 
lish, 0 otherwise. 



A continuous vector containing the number 
of days per week in which the subject's 
class was given bilingual instruction. 



A continuous vector containing the number 
of hours per day during which bilingual 
instruction was conducted. 



A continuous vector containing the number 
of hours per week during which bilingual 
instruction was conducted. 

A binary vector containing a 1 if the sub- 
ject's teacher holds a master's degree, 
0 otherwise. 

A continuous vector containing the number 
of years teaching experience of the sub- 
ject's teacher. 

A continuous vector containing the rating of 
the teacher's fluency in Spanish. 

A binary vector containing a 1 if a teacher 
aide was used in the subject's Claris, 0 
otherwise. 
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Table 10 (contiaued) 



Name 


Definition 


Predictors (continued) 


Aide Spanish 


A binary vector containing a 1 if the teach- 
er aide speaks Spanish, 0 otherwise. 


Mexican-American 
percentage 


A continuous vector containing the percen- 
tage of Mexican-American children in the 
subject* s class. 


Class size 


A continuous vector containing the number 
of children in the subject's class. 


Vectors 


Included for Linearity Tests 


Pretest squared 


A continuous vector containing the square 
of the pretest scores of the subjects. 


Viewing frequency 
squared 


A continuous vector containing the square 
of the number of Carrascolendas programs 
viewed by the subject (ranges from 0 to 50) 


Educational TV 
squared 


A continuous vector containing the square 
of the nijunber of educational television 
programs, other than Carrascolendas, viewed 
by the subject. 


Grade squared 


A continuous vector containing the square 
of the subject's grade level. 



Bilingual instruction 

(days) squared A continuous vector containing the square of 

the niimber of days per week in which the 
subject's class was given bilingual instruc- 
tion. 
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Table 10 (continued) 



Name 



Definition 



Vectors Included for Linearity Tests (continued) 



Bilingual instruction 

(hours/day) squared A continuous vector containing the square 

of the number of hours per day during which 
bilingual instruction was conducted. 



Bilingual instruction 
(hours/week) squared 



Teacher experience 
squared 



Teacher Spanish 
squared 



A continuous vector containing the square 
of the number of hours per week during 
which bilingual instruction was conducted. 



A continuous vector containing the square 
of the number of years teaching experience 
of the subject's teacher. 



A continuous vector containing the square 
of the rating of the teacher's fluency in 
Spanish. 



Mexican- American 

percentage squared A continuous vector con 'raining the square 

of the percentage of Mexican-American chil- 
dren in the subject's class. 



Class size squared 



A continuous vector containing the square 
of the number of children in the subject's 
class . 
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variable may be more complex than ^\ siiv.plo linear rclations]\i p . 
For example, an increase of one unit on the measure of program 
viewing might be associated with an increase of one unit on the 
criterion of posttest score. This would be a simple linear rela- 
tionship. The equation for expressing a eimple linear relation- 
ship is: Y = a^U + aj^X + E (1) 

In this equation Y represents the criterion vector of 
posttest scores; X represents the predictor vector, i.*e,, n^omber 
of programs viewed; E represents a vector of error terms, that is 
the difference in the predicted posttest score and the observed 
score; and, a^^ is a numerical constant or weight applied to X. 
The regression constant, i.e., the Y intercept, is represented by 
the weighting constant a^ applied to U, a unit vector, i^.e. , a 
vector containing a "1" in every element. 

If, however, the relationship between posttest scores and 
the viewing measure is curvilinear then an increase of one unit 
of viewing would not, in general, be associated with an increase 
of one unit on the posttest. In the case of a curvilinear rela- 
tionship, the equation would include a term v;hich represented the 
square of the predictor variable, as: Y = a^U a^X + b (X*X) -f E (2) 

In model 2 the square of the predictor variable is indi- 
cated by (X*X) , which follows computer notation using an to 
indicate multiplication. The relationship of predictor variables 
with the criterion variable may be tested to determine whether 
a simple linear relationship provides adequate prediction, or 
whether a curvilinear relationship is significantly more 

O 
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appropriate. Linearity tests were conducted for all of tlio con- 
tinuous variables in the predictor set in the following manner. 
Two models in the form of models 1 and 2 above were constructed 
using each continuous variable in turn to predict posttest scores. 
For example, pretest scores were used to predict posttest scores 
in the following models: 

Posttest = (a^) Pretest + (a^) (Pretest*Pretest) + Error (3) 

Posttest = (b^) Pretest + Error (4) 
If the curvilinear model (3) describes the data relationships 
more effectively than the straight line model (4), an F-ratio 
comparing the two models gives an indication of this difference. 
F-ratios comparing models of these forms, therefore, were calcu- 
lated for all continuous variables. Where significant F-ratios 
were obtained, the square of the predictor was also included in 
the starting model or equation for predicting the posttest scores. 
The results of these tests are presented in Table 11. 

As shown in Table 11, the curvilinear form of the equa- 
tion was significantly different from the linear or straight line 
form for five predictor variables when Spanish History/Culture 
posttest scores were used as the criterion. The starting model 
for the prediction of Spanish History/Culture scores, therefore, 
included the squared terms for these five variables. 

It should be noted that higher order curvilinear relation- 
ships, such as cubic, and interaction among predictor variables 
might also have been tested. The inclusion of such complex 



Table 11 



F- tests for Curvi linearity in 
Spanish History/Culture Predictors 



Variable Name 


RSQ 
Curvilinear 
Form 


RSQ 
Straight 
Line Form 


F-ratio 


Pretest 


.4502 


. 4463 


1.43 


Viewing frequency 


.0376 


.0376 


.01 


Educational TV 


.0015 


.0004 


.23 


Grade 


.0775 


.0727 


1.03 


Bilingual instruction (days) 


.2118 


.2087 


.79 


Bilingual instruction 
(hours/day) 


. 1702 


.1394 


7.46* 


Bilingual instruction 
(hours/week) 


.1453 


.1286 


3.91* 


Teacher experience 


.0381 


.0193 


3.93* 


Teacher Spanish 


.0511 


.0500 


.22 


Mexican-Airarican percentage 


.1320 


.0849 


10.90* 


Class size 


.0397 


.0003 


8.25* 



* p<.05 
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variables might have provided a better "fit" of the equation to 
the data of the prevalidation ^?wi»tjle. This increase in fitting 
the equation to the data would dsc have increased the sensitiv- 
ity of the equation to the peculiarities and idiosyncrasies of 
the prevalidation sample. In the present study, a more generalize 
able model was sought which would have high efficiency in cross- 
validation. Ward (1953) has shown that the inclusion of complex 
variables in equations constructed with data from the prevalida- 
tion sample tends to result in greater loss of predictive effi- 
ciency on cross-validation than the use of noncomplex variables. 
Higher order relationships and interactions, therefore, were not 
examined in the present study. 

Model Construction 

All phases of model construction in this study were ac- 
complished using linear multiple regression analysis. The analy- 
sis procedure was essentially one of model-seeking. A starting 
model incorporating all potential predictors including curvilin- 
ear expressions where appropriate, was calculated with a computer 
program which used an iterative process (Ward & Jennings, 1973, 
p. 317) . Such a process begins by setting all predictor weights 
at 0, and then adjusting the weight on one predicr.or at a time 
until some arbitrary criterion is met to stop the iteration pro- 
cess. In this study the stop-criterion was that the increase in 
the square of the multiple correlation coefficient (RSQ) had to 
be equal to or greater than .0001. The value of RSQ gives an 
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indication of the proportion of variance in the criterion measure 
which is being accounted for by the predictors in the model. The 
computer program begins by selecting the predictor variable which 
yields the greatest RSQ, and then adding a new variable froi the 
predictor set or adjusting the weight of a predictor already in 
the equation at each step in the iteration process. Table 12 pre- 
sents the first 30 iterations in the computation of the equation 
for the starting model. Below the iteration sequence final val- 
uers are reported for RSQ, the error sum of squares, the error 
mean square, and the standard error of the estimate. Inspection 
of the results calculated at each iteration step can serve as a 
rough guide to the way the model changes with the introduction of 
each new predictor variable. The error mean square is oZ parti- 
cular interest, since this statistic gives an indication of the 
predictive accuracy of a model. 

Next, in Table 13, all of the variables in the predictor 
set of the starting model are listed along with their raw weights, 
standardized weights, and correlation with the criterion vector. 
The column headed "Entry" is the order in which the predictors 
entered the equation during the iterative process. In general, 
J^e order in which predictors entered the equation is highly cor- 
related with the rank-order of measures of the "relative contrir 
bution" of each predictor to the prediction of the criterion 
(Ward & Jennings, 1973, p. 271). In the pres nt study the entry 
order of predictors was used as a guide in the construction of 
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Table 13 

Spanish History/Culture Starting Model 



Correlation 



Variable 


Raw 
Weight 


Standardized 
Weight 


with 
Criterion 


Entry 


Viewing frequency 




.0320 




. 398040 


. 1939 


3 


Activities (any) 


— 


.2206 


— 


.114627 


.0990 


17 


Activities (Spanish) 


- 


.0227 


- 


.013539 


.2733 


21 


Activities (English) 


— 


.4158 


- 


.239736 


-.0522 


5 


Color TV 




.1588 




.080315 


.1949 


14 


Sex 




.0489 




.029148 


.1050 


18 


Teacher r4A 




.2450 




.102019 


-.1406 


4 


Teacher experience 




.0087 





.091600 


-.1389 


8 


Teacher Spanish 




.0798 




.102414 


.2237 


10 


Aide 


- 


. 3326 


- 


.195204 


.1291 


6 


Aide Spanish 




.1141 




.068003 


.1194 


15 


Bilingual instruction 
(days ) 




.0296 




.082588 


. 4bbo 


ti 


Bilingual instruction 
(hours/day) 




.1060 




.295434 


.3733 


1 


Bilingual instruction 
(hours/week) 


e 


.oooo 


0 


.000000 


.3587 


0 


Percentage Mexican- 
American 


0 


.0000 


0 


.000000 


.2914 


0 


Class size 




.0054 




.038550 


.0172 


16 


Seseune Street 




. 3247 




.135212 


.0540 


11 


Grade 




.0844 




.082463 


.2967 


13 


Educational TV 




.1432 




.170394 


-.0202 


12 


Teacher experience 
squared 




.0001 




.039676 


-.1738 


20 


Bilingual instruction 
(hours/day) squared 


0 


.0000 


0 


.000000 


.3216 


0 



Table 13 (continued) 



Variable 


Raw 
Weight 


Standardized 

Weight 


Correlation 
with 
Criterion 


Entry 


Bilingual instruction 
(hours/week) squared 


-.0003 


-.107633 


.3182 


19 


Percentage Mexican- 
American squared 


.0001 


.263998 


. 3270 


9 


Class size squared 


0.0000 


0.000000 


-.0454 


0 


Pretest 


.8256 


.532019 


.6681 


1 


U: it Vector 


.2471 
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restricted models, as discussed below. (For reference, the means 
standard deviations , vari^ance, and correlation matrices of all 
variables in the predictor and criterion sets for both the preval 
idatior) and cross-validation samples are reported in Appendix C.) 

Visual inspection of the error mean square at each step 
of the iteration process (Table 12) reveals that entries in the 
last five steps of the iteration process being to fluctuate and 
are all greater than the lowest error mean square of .2817 which 
resulted at iteration step number 25. As can be seen in Table 12 
the values of the error mean square and RSQ at iteration 25 are 
quite close to the final values attained with all the predictors 
in the starting model. This fact, considered in conjunction with 
the fluctuation or the error mean square in subsequent iterations 
suggests that predictors added to the equation after step 25 con- 
tribute little additional predictive power or accuracy. Select- 
ing the predictors incorporated in the equation at a point a few 
steps beyond iteration 25 should ensure that the most useful pre- 
dictors are included in the restricted models. The variables 
which had been selected from the predictor set at iteration step 
28 were, ther'^fore, selected as the predictor set for the largest 
restricted model. Other restricted models were constructed by 
successively dropping out predictor variables in the reverse or- 
der of their entry into the equation for the starting model. 
This procedure resulted in 14 restricted models with predictor 
sets as shown in Table 14. These models were subjected to 
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Table 14 

Spanish History /Culture 
Restricted Models 



Predictor 

Variables 1 


2 


3 


4 


5 


Model 
6 7 8 


Number 
9 10 11 


12 


13 


14 


J 

w 

F-ratio** 


Pretest * 










★ * ★ 


* ★ * 








81.4149 


Bilingual instruction 
(days) 


* 


* 


* 


* 


★ * ^ 


* * * 






* 


71*9509 


''iewing frequency 




* 


* 


* 


* * * 


* * * 


if 


* 


* 


58*9082 


Teacher MA 






* 


* 


* * * 


* * * 


* 


* 


* 


50.4128 


Activities 
(English) 








* 


* * * 


* * * 




* 


* 


46.0120 


Aide 










* * * 


* ★ * 


* 


* 


* 


40.2964 


Bilingual instruction 
(hours/day) 










* * 


* * * 


* 


* 


* 


37.2284 


Teacher experience 










* 


* * * 


* 


* 


* 


33.7283 


Mexican-American per- 
centage squared 












* * * 


* 


* 


* 


31.3792 


Teacher Spanish 












* * 


* 


* 


* 


28.9232 


Sesame Street 












* 


* 


* 


* 


26.5328 


Educational TV 














* 


* 


* 


24.9777 


Grade 
















* 


* 


23*4137 


Color TV 


















* 


22.0295 




♦Indicates predictor variable was included in model. 
**A11 F-ratios significant (p<*01) . 
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F-tests to determino whether each, model yielded prediction sig- 
nificantly different from chance. Resultant F-ratios are also 
reported in Table 14, whore it can be seen that all models 
yielded F-ratios significant at an alpha level less than .01. 
At this point, 14 models had boon ccnstructcd, al] of v;hio]i 
yielded predictions of the criterion significantly better than 
chance. These models, furthermore, were expected to include an 
optimum model for the prediction of Spanish History/Culture 
scores. The number of possible models under consideration had 
been reduced in the first six steps of the research strategy to 
14 models. 

Cross-validation 

The final step in the research strategy was to select 
a single model as the optimum model. Among the fourteen models, 
some of the models, as noted above, may have been "over fitted" 
to the prevalidation data sample. On the other hand, some of 
the models did not provide the maximum predictive power possi- 
ble, as indicated by the relative size of RSQ. The optimum 
model was defined, therefore, as that model which provided a 
maximum RSQ, when applied to the data of the cross-validation 
sample. The selection of such a model avoided most of the prob- 
lem of "overf itting , " while also maximizing predictive power. 

The 14 restricted models were subjected to cross-vali- 
dation in two ways. First, a least squares solution was computed 
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for each model using data from the prevalidation sample. The 
resultant solution was applied to the data from the cross-vali- 
dation sample to produce a cross-validated RSQ. Second, a least 
squares solution was computed for each model using data from 
the cross-validation sample- These resultant solutions wore 
applied, in turn, to the data of the prevalidation sample. The 
RSQ's obtained from these computations are presented in Table 
15. As can be seen in the table, where the least squares weights 
from the prevalidation sample were applied to the data of the 
cross-validation sample, the cross-validated RSQ attained a max- 
imum value (.6849) using the predictor set of model 8. Cross- 
validated RSQ's using least squares weights from the cross-vali- 
dation sample attained a maximum value (.5874) using the predic- 
tor sets from models 13 and 14. Since model 8 is more parsimon- 
ious than models 13 or 14, in that model 8 incorporates fewer 
predictors, the properties of this model were further investi- 
gated. 

Optimum Model 

The least squares solution to model 8 using data from 
the prevalidation sample produces an equation which is considered 
to be the optimum model for predicting Spanish History/Culture 
posttest scores. The predictors, weights, and summary statis- 
tics for this model are presented in Table 16. The raw weights 
give a direct indication of the influence a particular predictor 
has upon the criterion. For example, the raw weight for viewing 
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Table 15 

Svanish History/Culture 
Cross-validations^ 





Cross-validating from 


Cross-validating from 




Sample 


1 


to Sample 2 


Sample 


2 


to Sample 1 




Sample 


1 


Sample 2 


Sample 


2 


Sample 1 


Model 


LSSb 




CV^ 


LSS° 




CV^ 


Niunber 


RSQ 




RSQ 


RSQ 




RSQ 


1 


.4463 




.6105 


6105 




.4463 


2 


.5178 




.6240 


.t374 




.5057 


3 


.5409 




.6539 


.6679 




.5284 


4 


.5588 




.6579 


.6759 




.5436 


5 


.5821 




.6609 


.6803 




.5578 


6 


.5887 




.6583 


.6808 




.5623 


7 


.6029 




.6833 


.7100 




.5745 


8 


.6087 




.6849 


.7108 




.5786 


9 


.6176 




.6789 


.7108 




.5786 


10 


.6217 




.6758 


.7110 




.5787 


11 


.6225 




.6778 


.7122 




.5770 


12 


.6291 




.6786 


.7141 




.5824 


13 


.6322 




.6773 


.7154 




.5874 


14 


.6350 




.6789 


.7154 




.5874 



Sample 1 was the prevalidation sample used for construc- 
tion of starting and restricted models. Sample 2 was the sample 
reserved for cross-validation. 

*^Least squares solution 

^Cross-validated 
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Table 16 



Spanish History/Culture 
Optimum Model 



Raw Standard- Correlation 

Variable Weight ized Weight with Criterion Entry 

Pretest .9205 .5932 .6681 1 

Bilingual instruction 

(days) .0450 .1255 .4568 2 

Viewing frequency .0297 .3701 .19 39 3 

Teacher MA -.3086 -.1285 -.1406 4 

Activities (English) -.3914 -.2257 -.0522 5 

Aide -.2214 -.1299 .1291 6 

Bilingual instruction 

(hours/day) .0582 .1623 .3733 7 

Teacher experience -.0065 -.0687 -.1389 8 



Unit Vector .2576 

Multiple Correlation 

Coefficient Squared (RSQ) = .6087 

Multiple Correlation Coefficient (R) = .7802 

Error Sum of Squares ^ 56.14 

Error Mean Sr^uare = .2879 

Standard Error of the Estimate = .5366 
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frequency is .0297. This weiijht indicates that an increase of 
.0297 in the posttest score is predicted for each unit increase 
in viewing frequency. For each additional Carrascolendas pro- 
gram viewed by the child, in other words, an increase of .0297 
in his posttest score would be predicted, other things being equal. 

The sign of the raw weight gives an indication of whether 
a particular predictor varieU3le represents an addition to the 
child's posttest score or a subtraction from the score, if all 
other predictor variables are held constant. For example, if 
bilingual instruction is conducted in the child's class, a high- 
er posttest score would be predicted than if bilingual instruc- 
tion is not conducted. On the other hand, if supplementary ac- 
tivities are conducted in English, a lower posttest score is 
predicted. 

The range of possible values taken on by a predictor 
variable should also be considered in evaluating the relative 
relationship of predictor variables to the posttest score cri- 
terion. For example, if the child viewed 27 Carrascolendas pro- 
grams, a posttest increase of .8019 would be predicted abo^^^e the 
score of a child who viewed none of the programs. In comparison, 
bilingual instruction cannot be conducted more than five days 
per week, nor more than six hours per day, assuming the school 
day for lower elementary children is six hours long. If bilin- 
gual instruction is conducted this maximum amount, then an in- 
crease of .5742 in the child's posttest score would be predicted. 
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Tl.^ relative values of these pi^edicted posttest increases are 
suggestive of the outcomes that might result from selecting one 
or the other of these activities to enhance the child's knowledge 
of history/culture. Table 17 presents the range of possible val- 
ues and the maximum increase (or decrease) in posttest score as- 
sociated with each of the predictors in model 8. In the case oi 

0 

teacher experience r the range of values actually observed in the 
data set has been reported in the table. 

If desired f predicted gain scores can also be computed 
from the information given in the model. For example, it may be 
of interest to know the predicted difference in gain scores for 
two children who have different pretest scores, but who are equal 
with respect to all other predictor variables. For a child whose 
pretest score is 2.0, a posttest score of about 2.10 is predicted 
(Posttest Score = 1.9205 X Pretest Score of 2.0] + [.2576 X Unit 
Vector of 11). The gain score for this child is 0.10. For a 
child whose pretest score is 3.0 r a posttest score of about 3.02 
is predicted. The gain score for this child, then is 0.02. The 
difference in gain scores is 0.08. Other predictors may, of 
course, be inducted in the calculation. If the child whose pre- 
test score was 2.0 had viewed 27 programs, then a posttest score 
of about 2.90 is predicted. If the chilo whose pretest, score 
was 3.0 had only viewed 15 progr2uns, then a posttest score of 
about 3.46 s predicted. Here the difference in gain scores is 
0.44. Although it is difficult to visualize a multivariate 
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Table 17 
Spanish History /Culture 

Range o£ Values and Change in Criterion 
Associated with Predictors in Optimum Model 



Predictor 




Range of 
Values 


Maximum Chance in 

Criterion 
(Posttest Score) 


Unit vector 




1 


.2576 


Pretest 




1-4 


3.682 


Bilingual instruction 


(days) 


0-5 


.2250 


Viewing frequency 




0-27 


.8019 


Teacher MA 




0-1 


-.3086 


Activities (English) 




0-1 


-.3914 


Aide 




0-1 


-.2214 


Bilingual instruction 


(hours/day) 


0-6 


.3492 


Teacher experience 




0-38 


-.2470 
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picture of these results as more variables are added to the equa- 
tion , a siiiiple two-variable relationship is easily shown in graph- 
ic form. Figure 4 presents a graph illustrating the relationship 
of viewing frequency and predicted posttest scores in Spanish 
History/Culture . 

English Science Model ConstructiorV 

\ 

As ntated above r identical procedures were\used for mod- 
el construction for both criterion vectors. The saunb, basic set 

> 

of predictors r furthermore r was used in both models. The pre- 
test and pretest squared vectors were^ of course, those associ- 
ated with the criterion vector , posttest scores on the English 
subtest of Science in the following instance. 

Linearity Tests 

Linearity tests were conducted on the relationship of 
each continuous variable with the cr iter ion « As can be seen 
in Table 18 r the results indicated that the curvilinear form 
of the equation was significantly different from the straight 
line form for five predictor variables. Squared terms for 
these five variables ^ therefore ^ were included in the start- 
ing model for the prediction of English Science posttest 
scores. 
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Figure 4. Spanish History/Culture 
Posttest Score versus Viewing Frequoncy 
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Table 18 

F-tests for Curvilinearity in ' 
English Science Predictors 



Variable Name 


RSQ 
Curvili no;^r 

Form 


RSQ 
St r;^ 1 ah t 
Line Form 


F-ratio 


Pr*^=» tes t 


. 5072 


4782 


XX • \J ^ \j \j 


Viewing frequency 


.0231 


.0221 


.2032 


Educational TV 


.0599 


.0581 


.3784 


Grade 


. 1839 


.1678 


3.9667 


Bilingual instruction (days) 


.0227 


.0019 


4.2695 


Bilingual instruction 
(hours/day) 


.0037 


.0020 


.3350 


Bilingual instruction 
(hours/week) 


. 0031 


.0025 


.1275 


Teacher experience 


.0292 


.0094 


4.1110 


Teacher Spanish 


.0594 


.0005 


12 .5825 


Mexican- American percentage 


.0634 


' .0562 


1.5644 


Class size 


.0037 


.0005 


.6525 



*p<.05 
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Model Construction 

Again ^ a least squares solution for the starting model 
was calculated with a computer program which used an iterative 
process (Ward & Jennings, 1973, p. 317) . The stop-criterion for 
the iteration process was set at .0001. The first 40 iterations 
are presented in Table 19, where it can be seen that fluctuation 
in the error mean square beings at iteration 33, with the intro- 
duction of the vector for "Bilingual instruction (days) squared" 
into the equation. The introduction of the next new vector, 
"Teacher Spanish" also causes a rise in the error mean square 
(iteration 40) . The sixteen variables which had been selected 
from the predictor set at iteration step 40 were, therefore, 
selected as the predictor set to begin construction of restrict- 
ed models. The final form of the starting model is presented 
in Table 20. As expected, this equation is quite different from 
the equation of the starting model for Spanish History /Culture . 
In other words, each of the different subtest area, scores is 
best predicted from different predictor sets. Of particular in- 
terest in this study was the fact that viewing frequency was not 
included in the least squares solution of the starting model for 
predicting English Science scores. 

Restricted models were constructed by using the sixteen 
predictor variable set and successively dropping out predictors 
in the reverse order of their entry into the equation for the 
starting model. This procedure resulted in 16 restricted models 
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Table 20 
English Science Starting Model 



Raw 

Variable Weight 


QtandAirdi zpfi 
Weight 


Correlation 
with 
Criterion 


Entry 


Viewino ^rpcriipncv 


• 0000 


. 000000 


. 1487 


0 


Activities (any) 


.0000 


.000000 


.1946 


0 


Activities (Spanish) - 


.2592 


-.167943 


.1301 


9 


Activities (Enalish) 


. 0953 


.059737 


.1718 


13 


Color TV 


.4246 


-.233391 


- .0578 


4 


Sex 


.0260 


-.016839 


-.0.124 


17 


Teacher MA 


• 0000 


.000000 


-.0018 


0 


Teacher experience 


.0000 


. 000000 

. V/ V w w w w 


0968 




Teacher Spanish 


.1337 


-.186346 


.0231 


16 


Aide 


.3689 


-.235266 


.0223 


8 


Aide Spanish 


.6315 


.408983 


.0358 


7 


Bilingual instruction 
(days) 


.0000 


.OOOOOP 


.0439 


0 


Bilingual instruction 
(hours/day) 


.0000 


.000000 


-.0447 


0 


Bilingual instruction 
(hours/week) 


.0052 


.078834 


-.0499 


14 


Percentage Mexican- 
American 


.0039 


.216851 


.2370 


2 


Class size 


-.0023 


-.017510 


.0218 


18 
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Table 20 (continued) 



ERIC 



Variable 



Raw Standardized 
Weight Weight 



Correlation 
with 

Criterion Entry 



Ses2une Street -.3158 -.142893 

Grade .1888 .200456 

Educational TV .1429 .184872 

Teacher experience 

squared .0004 .145896 

Teacher Spanish 

squared .0589 .277140 

Bilingual instruction 

(days) squared -.0040 -.061902 

Grade squared .0000 .000000 

Pretest 1.5605 1.447537 

Pretest squared -.2082 -.86-881 

Unit vector -.3534 



,0057 
,4097 
.2411 

.1349 

.0829 

.0250 
.3876 
.6915 
.6432 



II 
6 
10 

3 

12 

15 
0 
1 
5 



with predictor sets as shown in Table 21. As shown by the asso- 
ciated F-ratios in Table 21 ^ all of the restricted models yielded 
predictions significantly better than chance at an alpha level 
less than .Ol. 

Cross-validation 

In order to select an optimum model from among the 16 
restricted models, each model was subjected to cross-validation. 
Cross-validation procedures were the same as those used with the 
Spanish History/Culture models. Again, the optimum model was 
defined as that model which provided the maximum cross-validated 
RSQ. Where the least squares weights from the prevalidation 
sample were applied to the data of the cross-validation sample, 
the maximum cross-validated RSQ (.5163) was attained using the 
predictor set of model 9 (Table 22). Cross-validated RSQ's using 
least squares weights from the cross-validation sample attained 
a maximum value (.6175) using the predictor sets from models 15 
and 16 (Table 22). Coincidentally , these results are similar to 
the results of cross-validations for the Spanish History /Culture 
restricted models. 

The values of the cross-validated RSQ's, going either 
from the prevalidation to the cross-validation saunple or vice 
versa, indicate that model 9 is the optimum model for predicting 
English Science scores. 

Optimum Model 

The predictors, weights, and other statistics for the 
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Table 21 






English Science 






Restricted Models 






Predictor 


Model Number 




Variables 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 


F-ratio 



Pretest ****** 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


92.5648 


Mexican-Amer ican 

percentage ***** 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


75.1417 


Teacher experi- 
ence squared * * * * 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


62.2275 


Color TV * * * 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


53.1809 


Pretest squared * * 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


46.9219 


Grade * 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


42.3402 


Aide Spanish 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


37.7700 


Aide 




* 


* 


* 


* 


* 


* 


* 


* 


* 


36.5218 


Activities (Spanish) 






* 


* 


* 


* 


* 


* 


* 


* 


33.3370 


Educational TV 








* 


* 


* 


* 


* 


* 


* 


31.8587 


Sesame Street 










* 


* 


* 


* 


* 


* 


29.5190 


Teacher Spanish squared 












* 


* 


* 


* 


* 


27.6680 


Activities (English) 














* 


* 


* 


* 


26.1365 


Bilingual instruction 
(hours/week) 
















* 


* 


* 


24.4317 


Bilingual instruction (days) 
squared 


















* 


* 


23.0927 


Teacher Spanish 




















* 


22.7155 



ERIC 



^Indicates predictor variable was included in the model 
**A11 F-ratios significant (p<.01). 



99 



Table 22 

Eiiglish Science • 
Cross-validations* 







Cross-validating from 
Sample 1 to Sample 2 


Cross-validating from 
Sample 2 to Sample 1 


Model 
Number 


Sample 1 Sample 2 
LSsb CV^ 
RSQ RSQ 


Sample 2 Sample 1 
LSS° CV*^ 
RSQ RSQ 



1 


.4782 


.4692 


.4692 


.4782 


2 


.5286 


.4714 


.4852 


.5169 


3 


.5545 


.4882 


.5021 


.5414 


4 


.5719 


.4913 


.5068 


.5532 


5 


.5864 


.4886 


.5113 


.5689 


6 


.5999 


.4997 


.5209 


.5816 


7 


.6058 


.5080 


.5297 


.5369 


8 


.6271 


.5143 


.5361 


.6040 


9 


.6318 


.5163 


.5403 


.6086 


10 


.6444 


.5061 


.5403 


.6086 


11 


.6480 


.5061 


.5399 


.6074 


12 


.6523 


.5011 


.5418 


.6104 


13 


.6574 


.5100 


.5476 


.6168 


14 


.6587 


.5111 


.5497 


.6160 


15 


.6611 


.5097 


.5504 


.6175 


16 


.6706 


.5032 


.5504 


.6175 



Sample 1 was the prevalidation sample used for construe 
tion of starting and restricted models. Seunple 2 was the sample 
reserved for cross-validation. 

'^Least squares solution 

^Cr OS s - va 1 ida ted 
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optimum model are given in Table 23. The raw weights ^ along 
with their signs ^ indicate the influence a particular predictor 
has upon the criterion. For example r the predicted posttcst 
score for children who view Carrascolendas on a color television 
is .3107 less than the score predicted for children who view the 
series in black-and-white. In contrast, a child in the second 
grade would have a predicted score .1571 higher than a child in 
the first grade , if other conditions were equal. It should be 
noted that some of the predictors can be considered only in sets. 
A two-unit increase in pretest score r for instance, implies an 
increase in posttest score of 2.72 units. Since "^Pretest 
Squared** is also in the predictor set, however, the predicted 
increase in posttest score should be 2.11 (2.72 - [(.1523) X (2)^]). 
The complete range of values and the maximum change in posttest 
score associated with each predictor is given in Table 24. 

The optimum models for predicting Spanish History/Culture 
and English Science posttest scores are quite different in terms 
of their predictor sets. Other than the appropriate pretest 
score, only the binary measure of whether or not a teacher aiae 
is present in the child's class appears in both predictor sets, 
with a negative weight in both cases. The measure of viewing 
frequency of Carrascolendas appears in the Spanish History/Cul- 
ture model, but not in the English Science model. The models 
have a similar number of predictors (8 and 9) , probably coinci- 
dentally. The predictive power and accuracy of both models, 
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English Science 
Optimum Model 





Variable 


Raw 

Weight 


Standard- 
ized Weight 


Correlation 
with Criterion 


Entry 


Pretest 


1. 3622 


1. 263613 


.6915 


1 


percentage 


.005j> 


.294606 


.2370 


2 


Teacher experience 
squared 


.0004 


. 163051 


.1349 


3 


Color TV 


-.3107 


-.170805 


-.0578 


4 


pretest squared 


-.1523 


-.629481 


.6432 


5 


Grade 


.1571 


. 166848 


.4097 


6 


Aide Spanish 


.4922 


. 318796 


.0358 


7 


Aide 


-.3793 


-.294606 


.0223 


8 


Activities (Spanish) 


-.1579 


-.102286 


. 1301 


9 


Unit Vector 


-.1786 









Multiple Correlation 

Coefficient Squared (RSQ) = .6 318 

Multiple Correlation Coefficient (R) = .7948 

Error Sum of Squares - 44,73 

Error Mean Square = .2 306 

Standard Error of the Estimate = .4802 
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Table 24 

English Science 
Range of Values and Change in Criterion 
Associated with Predictors in Optimum Model 



Predictor 

Unit Vector 
Pretest 

Mexican- American percentage 
Teacher experience squared 
Color TV 
Pretest squared 
Grade 

Aide Spanish 
Aide 

Activities (Spanish) 



Maximum Change in 
Range of Criterion 
Values (Posttest Score) 



1 


-.1786 


1-4 


5.4488 


0-100 


.5300 


0-1444 


.5776 


0-1 


-.3107 


1-16 


-2.4368 


1-3 


.4713 


0-1 


.4922 


0-1 


-.3793 


0-1 


-.1579 
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furthermore, appears to be about the same, with RSQ values of 
about .60. 
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Chapter IV 



Discussion 

Summary of the Study 

This study focused on the development of a quantitative 
model for the prediction of learning effects of a bilingual chil- 
dren's educational television program series. A further aim was 
to assess the modeling procedure used in the study. 

The stimulus program series consisted of 30 half-hour pro- 
grams in the third-year series of Carrascolendas , a Spanish-Eng- 
lish series designed to supplement bilingual classroom instruction. 
The series was based upon specified behavioral objectives grouped 
into the content areas of Self Concept, History/Culture, Spanish 
Language Skills, English Language Skills, Phoneme/Grapheme Rela- 
tions, Science, and Math. 

Learning effects were measured with pre and postseries 
criterion referenced tests, where test items represented a sam- 
pling of the behavioral objectives of the program series. Tests 
were administered in oral interviews conducted in both Spanish 
and English with 408 Mexican-American children in kindergarten, 
first grade, and second grade at nine test sites across the na- 
tion. Data on independent variables vere collected on measures 
of communication stimuli, individual characteristics, and 

104 
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contextual characteristics. These measures were completed by the 
children's classroom teachers and field testing staff. 

The predictive models developed in this study were con- 
structed through linear multiple regression analysis. The criter- 
ion variables were oostseries criterion referenced test scores 
and the predictor variables were the measures of communication 
stimuli and individual and contextual characteristics. The 
desired model was a multivariate model which would show the rela- 
tionship of learning effects to frequency of viewing the program 
series in combination with other independent variables such as 
individual and contextual characteristics. Such a model would 
permit not only description of the impact of program viewing on 
learning effects, but also description of the influence of the 
other factors in moderating the learning effects resulting from 
program viewing. It was highly desirable, therefore, that pro- 
gram viewing frequency be strongly associated with learning ef- 
fects* If viewing frequency was not in the model, then little 
or nothing could be said about the relationship of viewing fre- 
quency to other independent variables in the attainment of learn- 
ing effects. On the basis of prior contract evaluation research, 
scores on the Spanish subtest in the content area of History/Cul- 
ture and the English subtest of Science appeared to be the only 
dependent variables which were reliably and consistently related 
to program viewing. These two variables were, therefore, selected 
as the criterion variables for separate models developed through 
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linear multiple regression analysis, where predictor variables 
were the measures of communication stimuli and individual and 
contextual characteristics • 

The basic research strategy was one of model-seeking, 
where multiple regression analysis was used to reduce the number 
of predictor variables for incorporation in an optimum predictive 
model. As can be seen in Table 25, the optimum models for predict- 
ing Spanish History/Culture scores and English Science scores have 
nine and ten predictors respectively. Table 2 5 also presents sum- 
mary statistics for comparing the two models in the following dis- 
cu.^sion. 

Summary of the Research 
Questions and Results 

The properties of the optimum models constructed for the 
prediction of scores on the Spanish subtest of History /Culture 
and the English subtest of Science provide the basis for answer- 
ing thie research questions posed for this stUdy. 

Prediction of Effects 

The first research question was: To what degree can 
learning -effects be predicted upon the basis of characteristics 
of the instructional communication treatment, the individual stu- 
dent characteristics, and the contextual variables? In summary, 
the results indicate that learning effects can be predicted with 
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relatively high degrees of reliability and accuracy. The term 
"relatively" is a necessary qualification. Guilford and Fruchter 
(1973) point out that correlations of .8 are about the highest 
level "usually found for useful predictive instruments in psycho- 
logical and educational practice [p. 347]." The; coefficient of 
multiple correlation (R) for both the Spanish History/Culture and 
the English Science models is nearly .8. The square of this co- 
efficient (RSQ) , furthermore, is about .6 for both models, indi- 
cating that about 60% of the variance in criterion scores is ac- 
counted for by the predictor variables incorporated in the models. 
Finally^ the cross-validated RSQ's for both models are only 4% 
lower than the RSQ from a least squares solution to the model us- 
ing data from the cross-validation sample. The least squares so- 
lution yields the maximum RSQ for a given model and a given sam- 
ple of data. These values of R and RSQ indicate that predictions 
based on these models have a generally high degree of reliability. 

The accuracy of these models is best judged in terms of 
the reduction in errors of prediction based upon the models, com- 
pared with predictions made without the model. This reduction 
amounts to about 37% for both models. Taken at its face value, 
this reduction may not seem very high. However, in relative terms 
the reduction is quite high. For example, if the criterion (post- 
test scores) were predicted upon the basis of knowledge of pre- 
test scores alone, errors of prediction would be 26% less than 
predictions made without knowledge of pretest scores. This 
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percentage is increased again by almost half (to 37%) with the 
inclusion of the other predictor variables in the model. For 
prediction in education and psychology in general, a 37% reduc- 
tion in errors of prediction is quite substantial (Guilford & 
Fruchter , 1973) . 

Implications 

The second research question in the study was: If a re- 
liable model for predicting learning effects can be developed: 
(a) What are the practical implications for policy decisions on 
children's bilingual educational television? (b) What are the 
implications for theorizing about rhe communication effects of 
educational television? 

Implications for policy decisions . The most general im- 
plication of the results of this study is that policy decisions 
regardi'^g educational television must be formulated in terms of 
contextual characteristics, as well as program content and objec- 
tives. For example, the Carrascolendas series enhanced learning 
in History/Culture; but, in classrooms where a teacher aide was 
present, students did not generally achieve posttest scores as 
high as otherwise. In other words, the impact attributed to the 
program in one classroom may not generalize to another classroom. 
This finding, in itself, is not surprising. The third-year con- 
tract evaluation revealed that the effects of the program series 
were different in different sites (see Table 5, Chapter One). 
Although the contract evaluation did not analyze these site 
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differences, the present study provides some insight into the 
problem. Differences in teachinq staff and classroom activities 
not only influence learning effects, but, in soi^e instances, are 
more salient predictors of learning effects than frequency of 
viewing the program series (as in English testing in the Science 
content area) . These findings are probably of more practical im- 
portance to the teacher and educational administrator than to the 
producer of the program series. 

For the teacher or administrator, the models answer ques- 
tions such as: 

(1) Will the program series enhance learning? 
(Yes, in some areas.) 

(2) Will the series be effective, even when it is used in 
classrooms with no bilingual instruction program? 
(Yes» Again, however , only in certain content areas, 
particularly History /Cul tare * ) 

(3) Can the program series be used as a substitute for 
hiring a teacher aide? 

(A teacher aide may be very useful in other ways; 
however , the present study revealed that children 
in classrooms with a teacher aide made lower post'' 
test scores than otherwise. It must be remembered 
that tests were tied directly to the objective.'* of 
the program series. A teacher aide is apparently 
of no benefit, st least in the areas of History/ 
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Culture and Science,) 
(4) If the program series is viewed in the classroomr 
can the children benefit from watching some, but 
not all of the series? 

( Yes . Findings in the present s tudy ind icato that 
the more programs the children view, th** most they 
benefit. There may be a lower limit, where viewing 
is essentially of no benefit, but that contingency 
was not investigated in the present study , ) 
Answers to questions such as these are best understood, 
of course, in the context of the entire study. For example, it is 
tempting to conclude that viewing the program series in color has 
an adverse effect on Science posttest scores, since color TV has 
a negative weight in the optimum model for predicting English Sci- 
ence scorers (Table 25). This result, however, may be an artifact 
of the analysis. Only children in San Antonio and Tucson viewed 
the series on color television sets. The color TV predictor, 
therefore, may be more tied to the results in these two sites, 
than to the general effect of viewing the series in color. The 
childran in these sites generally had lower scores than in other 
sites. The color TV predictor may have a negative weight, there- 
fore, simply because it is differentiating the children in San 
Antonio and Tucson from children in other sites. 

Another, and perhaps more practical, problem in inter- 
preting the results in the English Science optimum model is that 
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frequency of viewing the Carrascolendas series is not included 
in the predictor set. It is difficult to accept the notion that 
viewing the series in color detracts from posttest achievement, 
when there is no salient relationship between posttest scores and 
viewing the series at all. Again, it seems more appropriate to 
conclude that the color TV predictor represents a difference in 
site results, rather than true differences in program viewing 
conditions. 

The color TV predictor is the only predictor in either of 
the optimum models (Table 25) that is so tied to particular sites. 
It should be noted, however, that the teacher MI\ predictor repre- 
sents a variable that was not very well sampled, i.e., only about 
15% of the teachers reported having an MA degree. Effects attri- 
buted to this variable, therefore, should be regarded with some 
caution. In general, the remaining predictors in both models do 
not suffer from the limitations of interpretation noted in regard 
to color TV and teacher MA. 

One of the most interesting findings in the present study, 
for example, is that supplementary activities conducted in the 
language opposite that in which program materials were presented 
and tested appear to result in lower posttest scores. In the op^ 
timum model for predicting History/Culture scores in Spanish test- 
ing (see Table 25) , supplementary activities conducted in English 
is a salient predictor with a negative weight. Likewise, in the 
model for predicting Science scores in English testing (see Table 



25), supplementary activities in Spanish has a negative weight. 
About 50% of the children in this study participated in supple- 
mentary activities conducted in Spanish, with about 60% partici- 
pating in activities conducted in English (Appendix C) - Appar- 
ently when program materials are presented in one language and 
activities in the opposite language, a linguistic conflict occurs 
which produces confusion rather than enhancing learning of the 
program content. 

A similar problem may exist when a teacher aide is pres- 
ent in the cj^assroom. This predictor is the only predictor which 
is salient for both the prediction of Spanish History/Culture 
and English Science scores. In both models the aide predictor 
has a negative weight. This is somewhat surprising, since one 
would generally assume that an aide would provide more individ- 
ual attention to the viewer child. Such attention should enhance 
learning and posttest scores. Why this does not appear to be the 
case is open to question, but, perhaps, there is some confusion 
or conflict between information presented by the program or teach- 
er and information presented by the aide. Further research is 
necessary to resolve this question . 

There were, of course, limitations in this study. The 
possible value of repeated viewing of programs, for example, could 
not be analyzed. Other than differences in pretest scores, there 
were only two measures of individual differences, sex and grade 
level. In the present study, effects in Spanish History/Culture 
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were general izable across both sex and grade level, while effects 
in English Science were generalizable across sex, but not across 
grade level. Further investigation of the influence of other in- 
dividual differences would be desirable. More detailed measures 
of contextual characteristics, such as supplemental activities, 
would also be desirable. 

For producers or administrators in charge of program fund- 
ing, the greatest value of this study comes from the evaluation 
procedure utilized. The construction of linear models has yielded 
greater insight into the effects of the program series than other 
types of analyses have offered. For example, the third-year con- 
tract evaluation only showed that there were significant viewer- 
nonviewer differences favoring viewer gains in Spanish testing in 
the History/Culture content area. The present study, in contrast, 
reveals that increased frequency of viewing results in increased 
learning in the History /Culture area, that the magnitude of this 
benefit is at least equal to the benefit of a general program of 
bilingual instruction, and that increases in the amount of bilin- 
gual instruction results in increased posttest scores. Other 
salient predictors indicate that children generally make lower 
posttest scores when they participate in supplementary activities 
conducted in English or when they are in classrooms with a teacher 
aide or with more experienced teachers. (Perhaps teachers who 
have taught longer have more difficulty incorporating educational 
television into their curricula.) 
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The construction of linear models may also clarify re- 
sults obscured in other types of analysis. English posttest 
scores in Science were significantly and positively correlated 
with frequency of viewing the program ser ies This correlation 
seemed to confirm the results of the third-year contract evalua- 
tion, where significant viewer-nonviewer differences were found 
in English testing in the Science area. The present analysis, 
however, indicates that viewing frequency had little, if any, af- 
fect on posttest scores beyond that attributable to other factors. 
In other words, the program series had little if any unique affect 
on Science scores. A problem in conducting research on Carrasco - 
lendas is that the series is designed to supplement rather than 
replace bilingual curricula. In some content areas, the series 
may present information that is highly redundant with classroom 
instruction. The unique effects of the series in these areas will 
consequently be quite small. Science may be such a content area. 
In contrast, the series appears to have significant, unique ef- 
fects in the content area of History/Culture. Whether or not an 
educational television series is to be evaluated in terms of unique 
effects should be decided and specified early in the initial de- 
sign of the series. 

Unfortunately, the data analyzed in the present study did 
not include measures of many variables that are of great interest 
to program producers and funding agencies. Fiscal questions, for 
example, center on the relative cost--ef f ectiveness of different 
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production techniques. The relative effectiveness of different 
production techniques, such as animation or dramatic skits, hov- 
ever / was not investigated in this study. Although program pi'o- 
ducers may bo enl ightened by the findings of the present study , 
answers to their central questions of production cost-effective- 
ness must await further research-"^ 

Overall, the construction of linear models through the use 
of multiple regression analysis has proven to be a useful tool for 
multivariate analysis of the effects of educational television. 
While the results of the present study may be of greatest value 
to potential users of educational television, the evaluation tech- 
nique utilized has potential value for producers and fund admini- 
strators engaged in formative or summative research - 

Implications for communication theory . This study was 
an implementation of a multivariate view of communication- The 
results of this study confirm the view that a communicatio.^. stim- 
ulus, such as an educational television program series, functions 
as one of a variety of stimuli in producing learning effects- 
Relatively accurate, reliable predictions of effects can be made 
when contextual characteristics are considered in conjunction 
with characteristics of the communication stimulus- Past 



A f iscal analysis of Carrascolendas is forthcoming in F - 
Korman/ Toward a model of cost-effectiveness analysis for educa - 
tional television - Unpublished Ph-D- dissertation. University of 
TexdS at Austin, 1973- 
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theorizing on the multivariate nature of communication effects 
has been inadequate in specifying the magnitude of effects and 
the relative importance of the communication stimulus in the at- 
tainment of effects. This may have been due, in part, to a re- 
liance on statistical techniques which did not directly yield es- 
timates of relative magnitudes and on verbal-pictorial models of 
communication which were not easily operationalized in quantita- 
tive studies. The most important implication of the present 
study is that models constructed with the use of multiple regres- 
sion analysis and the mathematical notation associated with this 
technique provide a conceptual framework which allows for direct 
mathematical operationalization, estimates of relative magnitudes 
as well as directions of effects, and a multivariate view of com- 
munication. 

Although the present study did not include investigation 
of complex relationships among the independent variables, this 
decision was based more on limitations of the data base than on 
limitations of the technique. Curvilinear relationships of pre- 
dictors to criterion scores were investigated, and a significant 
curvilinear relationship was found between English Science pretest 
and posttest scores* This result, along with the frequency of 
significant complex relationships generally found in research on 
mediated communication, implies that further research should be 
conducted in which complex relationships are studied. 

The simple additive relationships which were investigated 
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in this study provide results which confirm and expand previous 
theorizing on educational television. Previously it could be 
said that educational television was an effective medium for in- 
struction in some content arear:. This was confirmed in the pres- 
ent study. Furthermore, the present study provides initial an- 
swers to some of the questions raised earlier by Schramm (1962) 
regarding how educational television can be effectively articu- 
lated into the classroom environment. In the case of a bilingual 
program series, for instance, classroom activities should be con- 
ducted in the same language in which the program material is pre- 
sented. Where the series has an impact, effects appear to be in- 
dependent of the sex of the viewer. However, effects do not ap- 
pear to be independent of the type of classroom (bilingual or 
monolingual curriculum) , nor the background and type of teaching 
staff. These conclusions are summarized concisely in the linear 
model notation exhibited in Table 25. 

Suggestions for Further Research 

Future research on the learning effects of educational 
television could profit from the inclusion of more detailed mea- 
sures of independent variables. In the present study, the mea- 
sure of supplementary activities, for exaunple, was limited to 
whether activities were conducted, and, if so, whether the lan- 
guage of activities was Spanish or English. Numerous other vari- 
ables, mentioned above, such as production techniques, as well as 
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complex relationships among variables also need to be investi- 
gated. 

It is also strongly suggested that contracts for the 
evaluation of specific program series require that the evalua- 
tion be directed toward the construction of linear models. Eval- 
uation conducted in this manner offers considerable potential for 
insight into the actual effects of the program series. Such 
evaluation would also contribute significantly to the ultimate 
construction and integration of a body of theory on educational 
television. 



APPENDIX A 
INSTRUMENTS 



CAKRASCOlLNDAS III 
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Nombre 



EXAMEN EN ESPAf^OL [1:1] 
NQ.de idontif icacion 



.[2,3,U] 



Primer Examen [b:l] 



Encierrc uno de 
estos en un circulo 



Encierre un<j de 
ectos en un circulo 



Orden [6] 1 Primero Grado: [7] 1 Primero 
2 Segundo 2 Segundo 

3 Jardin Infantil 



Preguntas para que el 
niho se sienta c6modo 



£C6ino te llamas? 

^Cu&ntos anos tienes? 

^CuSntas perscnas hay en tu familia? 



cC6mo 8c llaman tu pap& y tu mami?* 


O O 0 

4> ^ rH 

%^ 0 o o 
U u u 

if G 0 U 

M Pk n 
18] U 3 2 1 


0 « 

/C3 O W 

Id 4^ fH 

m a 
H X M 

[91 3 2 1 


j^Cu&l es tu direcci6n?* 


[lOli. 3 2 1 


[11]3 2 I 


iQxii idiomas ballast 
espanol m ing'ia 


112]U 3 2 1 


[13]3 2 1 



3FES8 
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CARRASCOLENDAS III 



ENGLISH TEST [Is 2] 



Child's Name 



Pretest [5:1] 



I.D. Number 1 ' 



[2',3,4: 



CIRCLE ONE 



CIRCLE ONE 



Order: [6] 1 First 
2 Second 



Grade Level: 



[7] 1 First 

2 Second 

3 Kindergarten 



QUESTIONS TO ASK CHILD TO 
PUT HIM AT EASE 



What is your name? 
How old are you? 

How many people are in your family? 



Vtuit is your address?* 


c 
o 

m o 

4> H 
O 4* * 

u o u o 

0 U 9i u 

fiL4 tu fiL^ :m 
[8] 4 3 2 1 


XI Xi 

» m 

^ 'H 

O C 
to H ^ 

M s: CO 
[9] 3 2 1 


What are the names of your mother 
and father? • 


[10] 4 3 2 1 


[111 3 2 1 


What languages do you speak? * 


[12] 4 3 2 1 


[13] 3 2 1 



3FEEe 
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CARRASCOLENDAS III 



Nombre 



EXAMEN EN ESPAHOL [1:1] 



Nq . de identif icaci6n 



Segundo Examen [5:2] 



[2,3,4] 



Encierre uno de 
estos en un circulo 



Orden [ 6 ] 1 Primero 
2 Segundo 



Encierre uno 
estos en un circulo 



Grado : [ 7 ] 1 Primero 

2 Segundo 

3 Jardin Infantil 



Preguntas para que el 
nino se sienta c6modo 



iCSmo te llamas? 

iCuSntos anos tienes? 

iCuSntas personas \ay en tu faunilia? 





Perf ecto 
Inducido 
rare 1 ax 
Error ; nada 


O M 
« 0 Nft> 
0 >3 r-l 

H to 

to *H C 

w ac 


cQu€ idiomas hablas?* 
eepatol e inglSe 


[ 8 ] U 3 2 1 


[ 9 ] 3 2 1 


Pinta un tercio de la pelota. 
(pelota dividida en tercios) 


[10] U 3 2 1 


[11:0] 



3FES8 

ERIC 
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:AKKASroi,L^j:)AS 111 



ENGLISH TEST (1:2] 



Child's NaTTie 



I.D. Number 



Pcst-test [5:21 



CIRCLE ONE 



CIRCLE ONE 



Order: 



[6] 1 First 
2 Second 



Grade Level: [7] 1 Fir^t 

2 Second 

3 Kindergarten 



QUESTIONS TO ASK CHILD TO 
PUT HIM AT EASE 



What is your name? 
How old are you? 

How many people are in your family? 





Perfect 
Prompted 
Partial 
Wrong; none 


English 

Mixed 

Spanish 


What languages do you speak?* 


[8] U 3 2 1 


[9] 3 2 1 


Which of these names is the name of 
a place: armadillo t San Antonio , 
mosquito? 
San Antonio 


do] U 3 2 1 


[11] 3 2 1 



3FEL8 
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CARRASCOLENDAS SITE STATUS REPORT 



Please describe how you obtained your test school(s) and any prob- 
lems you have encountered as well as changes in your original 
plan. 



What procedures did you undertake to solve any problems or diffi- 
culties? 



When was pretesting completed? 

Did the local television station or cable company cooperate with 
your efforts? (Please describe) 



Please indicate the viewing status of the series. 

Subjects (K, 1, 2) watch the series in one classroom 

Subjects watch the series in different classrooms in the same 

school 

Subjects watch the series in different schools 

Other (please specify) 

How is the television station airing the show? That is, is the 
broadcasting Jcne under the auspices of PBS, cable tv, or were 
other axrangements made? 
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Please indicate below on which particular day each show will be 
televised. Be sure to note if any programs will be missed dur- 
ing Thanksgiving. Also, since post-testing begins two weeks to 
10 days before Christinas vacation, what programs will not have 
been seen when post-testing begins . ) Do this for programs 1-30. 

PROGRAM PROGRAM PROGRAM 

NUMBER DATE NUMBER DATE NUMBER DATE 

1 11 21 

2 12 22 

3 13 23 

»♦ !«♦ 2U 

5 15 25 

6 16 26 

7 17 27 

8 18 28 

9 19 29 

10 20 30 



When do you plan to begin post-testing? 

When will you begin the Parent Attitude Survey? 

What is the general atmosphere that you find yourself working in 
(congenial, tense, etc.)? In particular, do you foresee any poten- 
tial problems such as teacher strikes, etc? 
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School Name 

NAME OF CHILD ABSENCES (Month and Date) 



Grade Level 

(Name) (Dates of Absences) 



ERIC 



site identification 
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NONVIEWER CLASSROOM INFORMATION 



Naine:_ ^ ^ Grade Level: 

Number of 

Position: children in class: 



1. Do you speak Spanish? (Check one) 

fluently moderately limitedly English only 

2. What is your educational background? (Check one) 

^Teacher certificate, Elementary Education 

^Temporary teacher certificate. Elementary Education 

^Teacher certificate, other: 

other: 



(Check one) 

Bachelor degree in Elementary Education 

Bachelor degree in Spanish 

Bachelor degree in Bilingual Education 

Bachelor degree in Foreign Language Education 

Master's degree 

other: 



3. How many years have you taught school? 

Does the class receive any bilingual instruction? 

(Check one) (Give approximate totals) 

^^nce^a week { J^^^^ ^^V. 

twice a week ' ^^^^^ 

^three times a week 

four times a week 

5. Does the class participate in any televised instruction? yes no 

If yes, give name: 

6. Number of children in class participating in CARRASCOLENDAS oxpori- 
mcnt: 

Chihlt^Mi's idonti r ioat ion mimbor:5 in lltiu o l.iS!;iH>om: 



ERLC 



Site identification 



VIEWER CLAS3R00M INPORMATIOM j^29 



*anie: _ Grade Level: 



Position: liuraber of children in class: 



1. Do you speak Spanish? (Check one) fluently limitedly 

m oderately English only 

2. What is your educational background? (Check one) 

Teacher certificate. Elementary Education 

Temporary teacher certificate. Elementary Education 

Teacher certificate, other: 

other: 



(Check one) 

J Bachelor degree in Elementary Education 

^Bachelor degree in Spanish 

Bachelor degree in Bilingual Education 

^EAchelor degree in Foreign Language Education 

I toster's degree 

other: 



3. Hov many years have you taught school? 

Does the class receive any bilingual instruction? 

(Check one) (Give approximate totals) 

^^_jlally I hoiirs per day 

once a week ^ . 

twice a week * »»«^» 

t hree times a week 
^four times a week 

3* Does the class participate in any other televised instruction besides CARRASC0LENDAS7 
n o y es If yes, give name: 

6. What days do you view CARRASCOLENDAS? 

M onday T uesday W ednesday T hursday F riday 

7. What time do you view the series? 

3, Do you have a Teacher Guide for CARRASCOLENDAS? yes no 



9. Do you plan to initiate class activities about CARRASCOLENDAS before viewing the 
prot^ram? (Check one) a lways sometimes r arely never 

10. Do you plan to initiate class activities about CARRASCOLENDAS after viewing the 
program? (Check one) a lways s ometimes r arely never 

11. Will activities be conducted in Spanish or English? (Check one) 

m ainly Spanish h alf Spanish and half English 

m ainly Ehglish n o activities 

12. Number of children ^.n class participating in CARRASCOLENDAS experiment: 

13« Children's identification numbers in this classroom: 



ERLC 



GENERAL INFORMATION 
Please fill in the blanks below. 2 30 

Haiae School Name 

Address Address 



Social Security # Grade Level 



What is the ethnic composition of your class? (Approximate %) 
% Mexican-American % Black % Other 



% Anglo % Indian 100* TOTAL 

What percentage of the pupils viewing the program with you speak: 

SPANISH ENGLISH 

% Fluently % Fluently 

% Moderately % Moderately 

% Limitedly % Limitedly 

% Comprehend only % Comprehend only 

% None % None 

100* TOTAL 100* TOTAL 

Do you speak Spanish? Do you have a teacher aide (or team 

Fluently teacher) who speaks Spanish? 

Moderately No aide 

Limitedly Aide who does not speak Spanish 

No Spanish Aide who speaks Spanish 

What is the extent of Spanish instruction in yoixr school? (Please be 
specific* ) 



Has your class ever viewed a television series in class before? Y es N o 
Which one: ^ 
Are you viewing CAPRASCOLENDAS in color? Yes No 



Please write a brief description of the situation in which you view CARRAS- 
COLENDAS — how often viewed, at what hour, and under what conditions (type of 
room, number viewing, etc. Use back of sheet if additioncLL space is needed.) 



APPENDIX B 
INTERVIEWER RELIABILITY 
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Interviewer Reliability 

If you asked a child to color 1/3 of a circle (divided into 
thirds), and he colored 2/3 of the circle, you would score:* 

4 3 2 (2) *1 (II) NR=(3) 

When you ask the child "What is your address?", and he re- 
sponds with the name of the city; and, when you repeat the 
question again, he does not respond further, you would score: 

4 3 2 (2) *1 (14) 

If a child is being tested on an objective of the use of on , 
you ask him, "Where is the boy riding?" (visual of a boy on a 
bus), and he responds "to school" or "to the. store," and you 
ask, "Yes, but where is he?" and he responds, "iri the bus," 
you would score: 

4 3 2 (2) *1 (13) NR=(1) 

If to the same question (3) after following the same procedure, 
he responds by answering, "on the bus," you would score: 

4 (1) *3 (13) 2 r) 1 NR*(1) 

If you ask the child "What languages do you speak?", and he 
only responds, "English" (on further checking you discover 
that he does not speak Spanish), you would score: 

*4 (13) 3 2 (2) 1 (1) 

If a child is asked the name of his parents, and he does not 
respond, you would score: 

Content: 4 3 2 1 (3) Language: 3 (3) 2 1 

OR 

♦Content: 4 3 2 1 (13) Language: No mark in language 

colmnn 

I conducted the test interview in (circle one) : 

Spanish (9) English (7) 



♦Correct score for each item. Number of interviewer responses 
indicated in parentheses » 



APPENDIX C 
STATISTICAL TABLES 



Variable List 
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ERIC 



!• Viewing froquoncy 

2. Activities (any) 

Activities (Spanish) 

4, Activities (Hnqlish) 

5. Color TV 
6 • Sex 

7. Teacher MA 

8. Teacher experience 

9. Teacher Loanish 
10 • Aide 

11 • Aide Spariish 

12. Bilingual' instruction (days) 

13. Bilingual instruction (hours/day) 

14. Bilingual instruction (hours/week) 

15. Mexican-American percentage 

16. Class size 

17. Sesame Street 

18. Grade 

19. Educational TV 

20. Viewing frequency squared 

21. Teacher experience squared 

22. Teacher Spanish squared 

23. Bilingual instruction (days) squared 

24. Bilingual instruction (hours/day) squared 

25. Bilingual instruction (hours/week) squared 

26. Mexican-American percentage squared 

27. Class size squared 

28. Grade squared 

29. Educational TV squared 

30. Spanish pretest Self Concept 

31. Spanish pretest Language Skills 

32. Spanish pretest History/Culture 

33. Spanish pretest Science 

34. Spanish pretest Total 

35. Spanish pretest Phoneme/Grapheme 

36. Spanish pretest Math 

37. Spanish posttest Self Concept 

38. Spanish posttest Language Skills 

39. Spanish posttest History/Culture 
^0. Spanish posttest Science 

41. Spanish posttest Total 

42. Spanish posttest Phoneme/Grapheme 

43. Spanish posttest Math 

44. English pretest Self Concept 

45. English pretest Language Skills 

46. English pretest History/Culture 

47. English pretest Science 
43. English pretest Total 

49. English posttest Self Concept 

50. English posttest Language Skills 

51. English posttest History/Culture 

52. English postt3st Science 

53. English posttest Total 

54. Spanish pretest History/Culture squared 
O . 55. English pretest Science squared 
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Prevalidation Sample 











Range of 


Variable 




Standard 




Observed 


Nurnber 


Means 


Deviations 


Variance 


Values 


1. 


18. 8039 


10 . 44 32 


±.\J ^ w \J H y \J 




2 . 


. 7451 


.4358 


. X O 7 7 




3. 


. 5000 


.5000 


.2500 


0-1 


4 . 


. 6275 


.48 35 


.2338 




5. 


. 2353 


. 4242 


. 1799 


n-1 


6. 


4755 


.4994 


.24 94 


n-1 


7- 


. 1422 


. 3492 


. 12 19 


0-1 


8. 


7. 3284 


8.8365 


78 0833 


0-38 


9. 


2.1373 


1,0759 


1 . 1576 


0-3 


10. 


. 5882 


. 4922 


.2422 


0-1 


11. 


.4853 


. 4998 


.2498 


0-1 


12. 


3.0490 


2 . 3405 


5 . 4780 


0-5 


13. 


1.9069 


2 . 3381 


. 'I 6 8 


0-6 


14. 


9.5392 


11.7370 


137,758^ 


0-30 


15. 


62.3627 


42.8737 


1838 , 1527 


22-100 


16. 


24.3922 


6.0052 


36.06J9 


14-33 


17. 


. 1422 


. 3492 


, l.*: L9 


0-1 


18. 


2.0049 


.8195 


.6715 


1-3 


19. 


.5931 


.9981 


.9962 


0-4 


20. 


462.6471 


272.4144 


74209 , 5813 


0-729 


21. 


131.7892 


320. 3143 


102601 .2742 


0-1444 


22. 


5. 7255 


3.6304 


13. 1795 


0-9 


23. 


14.7745 


11.9122 


141,9001 


0-75 


24 . 


9.1029 


13.9475 


194 , 5335 


0-36 


25. 


228.7549 


348. 4879 


121443, 8321 


0-900 


26. 


5727.2647 


4395. 0416 


19316390 , 3711 


0-10000 


27. 


631.0392 


221.3151 


48980 , 3710 


196-1089 


28 . 


4.6912 


3. 3133 


10 , 9 782 


1-9 


29 . 


1. 3480 


3. 1718 


10 0602 


0-16 


30 . 


2. 3534 


. 89 16 


- 7949 


1 00-4 00 


31. 


2.6140 


. 7737 


,5986 


1 no-? 87 


32 . 


1. 5999 


.5404 


.2920 


1 00-3 50 


33. 


2 . 1458 


. 8349 


. 6970 


1 00-4 00 


34 . 


2. 3756 


.6512 


.4241 


1 00-3 75 


35 . 


2 . 4299 


. 7901 


624 3 


1 03-4 00 


36 . 


2.6033 


. 8677 


. 7528 


1 00-4 nn 


37 . 


2 . 566L 


.8978 


,8060 


1 nn-4 nn 


38. 


2.8367 


.7688 


,5910 


1,00-3,81 


39. 


2.0699 


.8386 


. 7033 


1. jO-4 ,00 


40. 


2.2649 


.8341 


.6957 


1.00-4 ,00 


A 1 


Z . O 4 DU 


. b ob 1 


, 470 8 


1 . 0 5- 3 , 79 


42. 


2. 6591 


.8348 


.6969 


1,00-4,00 


43. 


2.9477 


.8558 


.7324 


1,00-4,00 


44. 


2.3606 


.7396 


• 5470 


1,00-4,00 


45. 


2.4983 


.6621 


.4384 


1.00-3,81 


46. 


1.7165 


.5814 


. 3381 


1.00-3,00 


47. 


2.2575 


.7159 


,5124 


1,00-3,59 


48. 


2,3130 


.5942 


. 3531 


1,07-3,52 


49. 


2.6827 


.8072 


,6516 


1,00-4,00 


50. 


2.7558 


.7002 


.4902 


1.06-3.81 


51. 


2.0100 


.5680 


, 3226 


1,00-3.50 


52. 


2.6037 


.7717 


,5955 


1,00-3,94 


53. 


2.6227 


.6402 


.4099 


1.25-3,68 


54. 
55. 


2.8516 
5.6087 


2.0030 
3.1902 


4.0120 
10.1774 


1.00-12.25 
1.00-12.88 
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Cro5;G-Val j dation Sample 



VcUTJ.clDXG 


PIC oil 5 


Standard 


Variance 


Range of 
Observed 
Values 


1. 


18.6471 


10.5442 


111. 1794 


0-27 


2. 


.7451 


.4358 


. 1899 


0-1 


3- 


.4951 


.5000 


. 2500 


0-1 


4. 


.6275 


. 4835 


.2338 


0-1 


5. 


,2304 


.4211 


.1773 


0-1 


6. 


. 4559 


. 4980 


.2481 


0-1 


7. 


.1569 


. 3637 


. 1323 


0-1 


8. 


7.5931 


9.0368 


81.6629 


0-38 


9. 


2.1618 


1.0610 


1.1258 


0t3 


10. 


.5637 


.4959 


.2459 


0-1 


11. 


. ^804 


.4996 


.2496 


O-l 


12. 


3.0245 


2.3461 


5.5043 


0-5 


13. 


1.8824 


2.3318 


5.4371 


0-6 


14. 


9. 4314 


11.6938 


136.7453 


0-30 


15. 


61.2617 


43.2706 


1872.3417 


22-100 


16. 


24.3088 


5.9448 


35.3409 


14-32 


17. 


.1373 


.3441 


.1184 


0-1 


18. 


1.9902 


.8224 


.6764 


1-3 


19. 


.5931 


.9932 


.9864 


0-4 


20. 


458.8922 


273.9051 


75023.9982 


0-729 


21. 


13^ . 3186 


328.0957 


107646.8151 


0-1444 


22. 


5.7990 


3.6372 


13.2292 


0-9 


23. 


14.6520 


11.9349 


142.4426 


0-25 


24. 


8.9804 


13.9171 


193.6859 


0-36 


25. 


225.6961 


347.7241 


120912.0251 


0-900 


26. 


5625.7059 


^412.1623 


19467175.8547 


0-10000 


27. 


626.2598 


216.9667 


47074.5649 


196-1024 


28. 


4.6373 


3.3190 


11.0155 


1-9 


29. 


1.3382 


3.1651 


10.1079 


0-16 


30. 


2.2912 


.9297 


.8644 


1.00-4.00 


31. 


2.5] 11 


.8581 


.7363 


1.00-3. 80 


32. 


1.6097 


.6081 


. 3698 


1.00-4.00 


33. 


2.0674 


.8413 


.7078 


1.00-4 .00 


34. 


2. 3180 


.7398 


.5473 


1.00-3.77 


35. 


2.3850 


.8822 


• 7783 


1.00 -4.00 


36. 


2.6062 


.9368 


.8777 


1.00-4.00 


37. 


2-4990 


.9791 


.9586 


1.00-4.00 


38. 


2.7739 


.8329 


.6937 


i. 00-3. 87 


39. 


2.0760 


.8675 


.7525 


1.00-4.00 


40. 


2.2721 


.9123 


.8322 


1.00-4.00 


41. 


2 .6029 


.7625 


.5814 


1.00-3.95 


42. 


2.5747 


.9268 


.8589 


1.00-4.00 


4 3 • 


2 .9493 


.9186 


. 84 39 


1.00-4 .00 


44. 


2.5139 


.7017 


. 4923 


1.00-4.00 


45. 


2.5827 


.5693 


. 3242 


1.19-3.88 


46. 


1.7475 


.5753 


.3310 


1.00-3.00 


47. 


2. 3224 


.6594 


.4348 


1.00-3.82 


48. 


2. 3932 


.5196 


.2699 


1.14-3.70 


49. 


2.7900 


.6395 


.4089 


1.00-4.00 


50. 


2.9066 


.6248 


.390 3 


1.19-4.00 


51. 


2.0327 


.5276 


.27«3 


1.00-3.50 


52. 


2.7047 


.6524 


.4256 


1.00-3.82 


53. 


2.7348 


.5325 


.2636 


1.02-3.73 


54. 


2.9609 


2.3196 


5.3804 


1.00-16.00 


55. 


5.8283 


3.1194 


9 .7305 


1.00-14.62 
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